Re: [Savannah-hackers-public] Problems with initial load of new Git repo

[David Hill]

Dear Assaf,

Many thanks for your detailed reply, and apologies for the delay in responding. 
I was out of my office for a day.

On Sep 9, 2015, at 7:52 41AM, Assaf Gordon wrote:

> Hello David and all,
> 
> On 09/08/2015 04:30 PM, David Hill wrote:
>> I am having problems uploading some of the relevant files/folders
>> into a new Git repo for the 'gnuspeech' project, prior to the first
>> official release. Karl Berry suggested I contact you.
> 
> To make sure we're talking about the same things, you are referring to this 
> project:
>   http://savannah.gnu.org/p/gnuspeech
> and this git repository:
>   http://git.savannah.gnu.org/gitweb/?p=gnuspeech.git
> 
> and no other repositories / projects, correct?
> that is, all the projects/folders/repositories you are mentioning below are 
> parts of this project?

Yes, that is correct.

> 
> If I see correctly, you've populated the repository just three days ago for 
> the first time.
> Were the files in that repository copied from another public repository (e.g. 
> an old CVS/SVN repo) ?
> If that is the case, I'd say we can safely reset the repository and have a 
> fresh start
> (normally, Savannah does not allow 'reseting' an existing long-established 
> repository).

Well, in effect, yes. Copied, modified, added to (specifically the manuals) and 
then, instead of returning the updated material to the SVN repo, we decided to 
move to a Git repo, which is what I tried to load, and botched so badly.

> 
> 
> Now, to the project's structure:
> 
> Looking at the git repository, there are indeed some strange entries there.
> There are also many binary entries that usually do not below in a source-code 
> repository.
> 
> For example, some output for 'git log --stat' shows:
> ====
> commit c5116c14e0d6718aace657237df8a5bf86001d0c
> Author: David Hill <address@hidden>
> Date:   Sat Sep 5 14:31:03 2015 -0700
> 
>    Initial set of files for release
> 
> gnuspeech/.DS_Store                                       |   Bin 0 -> 6148 
> bytes
> gnuspeech/Builds/.DS_Store                                |   Bin 0 -> 6148 
> bytes
> .../Contents/Frameworks/GnuSpeech.framework/GnuSpeech     |     1 +
> [...]
> .../Debug/CompilerIdCXX.build/CompilerIdCXX.hmap          |   Bin 0 -> 1561 
> bytes
> .../Objects-normal/x86_64/CMakeCXXCompilerId.d            |     2 +
> .../Objects-normal/x86_64/CMakeCXXCompilerId.dia          |   Bin 0 -> 220 
> bytes
> .../Objects-normal/x86_64/CMakeCXXCompilerId.o            |   Bin 0 -> 3624 
> bytes
> .../Objects-normal/x86_64/CompilerIdCXX.LinkFileList      |     1 +
> .../x86_64/CompilerIdCXX_dependency_info.dat              |   Bin 0 -> 14112 
> bytes
> .../Script-2C8FEB8E15DC1A1A00E56A5D.sh                    |     2 +
> .../CompilerIdCXX.build/Debug/CompilerIdCXX.build/dgph    |   Bin 0 -> 28828 
> bytes
> .../CompilerIdCXX/CompilerIdCXX.xcodeproj/project.pbxproj |   110 +
> gnuspeech/GnuspeechSA/build/CMakeFiles/CMakeOutput.log    |   696 +
> .../GnuspeechSA/build/CMakeFiles/TargetDirectories.txt    |     5 +
> gnuspeech/GnuspeechSA/build/CMakeFiles/cmake.check_cache  |     1 +
> gnuspeech/GnuspeechSA/build/CMakeFiles/feature_tests.bin  |   Bin 0 -> 8784 
> bytes
> [...]
> .../Objects-normal/x86_64/DictionarySearch.d              |     5 +
> .../Objects-normal/x86_64/DictionarySearch.dia            |   Bin 0 -> 220 
> bytes
> .../Objects-normal/x86_64/DictionarySearch.o              |   Bin 0 -> 154652 
> bytes
> .../Objects-normal/x86_64/DriftGenerator.d                |     3 +
> .../Objects-normal/x86_64/DriftGenerator.dia              |   Bin 0 -> 220 
> bytes
> .../Objects-normal/x86_64/DriftGenerator.o                |   Bin 0 -> 5532 
> bytes
> [...]
> .../build/Debug/TRAcT.app/Contents/Info.plist             |    50 +
> .../build/Debug/TRAcT.app/Contents/PkgInfo                |     1 +
> .../build/Debug/TRAcT.app/Contents/Resources/00-gray.tif  |   Bin 0 -> 27592 
> bytes
> .../build/Debug/TRAcT.app/Contents/Resources/01-gray.tif  |   Bin 0 -> 27592 
> bytes
> .../build/Debug/TRAcT.app/Contents/Resources/02-gray.tif  |   Bin 0 -> 27592 
> bytes
> .../build/Debug/TRAcT.app/Contents/Resources/03-gray.tif  |   Bin 0 -> 27592 
> bytes
> .../build/Debug/TRAcT.app/Contents/Resources/04-gray.tif  |   Bin 0 -> 27592 
> bytes
> ====
> 
> Some of these are compiled binary files (from a Mac?) that should not be 
> stored in a git repository.

My lack of experience. I thought the .gitignore files would take care of not 
uploading extraneous material. Worse, I overlooked the .git entries. The .git 
entry under GnuSpeech, which I cloned from a colleague was, it turns out, a 
complete Git repo, and a good reason the upload failed for that material. I am 
sadder and a little wiser.

> 
> I would recommended starting over with a clean repository, and carefully 
> adding only source code files (and only files under the 'gnuspeech' source 
> code directory).

I definitely think starting over is the way to go. I apologise for not being 
better informed. Incidentally, I thought I had isolated what needed to be 
loaded from the junk that shouldn't be loaded, but obviously did not test my 
assumption thoroughly. I expect to do it properly in the future, and will take 
your helpful advice on procedures and testing before trying to hit up the 
savannah-based Git repo again.

You mention "binary entries". Are "builds" not acceptable if they are not done 
under GNU/Linux?

I was all too aware of the fact that repos on savannah are not to be altered, 
and appreciate your significant concession on that. 

> 
>> My colleague suggested it would be better to have six different repos
>> in the project, one for each topic folder in the existing folder,
>> rather than one Git repo.
> 
> We can certainly create six sub-repositories under the main gnuspeech, and 
> you could use them as 'git submodules'.
> 
> If you want that, let me know their names (I'd assume the folders you've 
> mentioned, but best to verify).

Yes, I have been having some discussion on that, and it may be that fewer would 
suffice -- four to be precise. A "Sources" submodule for GnuSpeech and TRAcT, a 
"GnuspeechSA" submodule for the CMake cross-platform version, and a 
"Documentation" submodule for the manuals and any other documentation (I don't 
need to include a full text of the GPL and FDL, I presume, provided they are 
referenced). If builds are permitted, I would also include a fourth submodule 
"Builds". I could also bring the number up to six again by bringing the 
"gnustep" and "nextstep" branches into the Git repo. Perhaps this is not 
necessary.

> 
>> Having said that, I don't understand why four of the folders loaded
>> as expected, but two didn't. The two that loaded (GnuspeechSA and
>> TRAcT manual) are quite similar in form to the two that didn't
>> (GnuSpeech and monetManual).
> 
> The git repository contains strange enough entries, I'd not try to guess how 
> or why at this point.
> 
>> I would be grateful for your comments/advice. I am ready to make our
>> first official release, and this is obviously a hold-up.
> 
> I'd suggest the following:
> 
> Locally (on your computer):
> 1. create a new git repository with gnuspeech's files.
> If you prefer to import an existing git repository, carefully ensure it only 
> contains relevant files (and not binary/compiled files)
> 2. If you want to use git-submodules, create a new git repository for each 
> submodule,
> and use 'git submodule' to add the submodules.
> More information here: 
> https://git.wiki.kernel.org/index.php/GitSubmoduleTutorial
> 
> Test the local repositories by cloning them to a different location (or a 
> different computer) and ensure all needed files are there.
> 
> Remotely:
> Savannah is not a good place to experiment. There is a strict policy of not 
> deleting files/repositories once they are established,
> thus, fixing git mistakes is not easy.
> I would humbly suggest using a different git service for experimentation 
> (e.g. you could install 'gitolite' locally, or use other publicly available 
> services).
> Experiment with 'git push' of your git repository (and its submodules), until 
> you have it working the way you want.
> 
> Once you are pleased with the state of your git repository,
> we can reset the gnuspeech git repository (as an exception to the 'no code 
> removal' rule, because you've just created it),
> and you could push your updated repository there.
> 
> 
> Please remember:
> Once you've pushed some code into the repository (and certainly after you've 
> made an official release),
> we can not remove code or reset a git repository or fix any commit/push 
> mistakes - the only way to fix those
> is to commit+push another change reverting the mistake.
> 
> 
>> The other problem is that I failed to GPG-sign any of the files, but
>> assume that can be remedied if I can upload all the files, having
>> signed them.
> 
> I'm not sure what you mean by "failed".
> 
> To upload files to the git repository in Savannah there is no need to 
> GPG-sign anything.
> You already have an SSH public-key setup and you can push to the git 
> repository.

I think I became disoriented!

> 
> GPG-signing is required for another task: uploading a new tarball of an 
> official release to ftp.gnu.org.
> You should follow the instructions here to sign an official release:
> https://www.gnu.org/prep/maintain/html_node/Automated-Upload-Registration.html
> If a new upload is not signed with your valid key, it will not appear on the 
> official 'ftp.gnu.org' server.
> However,
> please note that this GPG check and FTP uploads are not handled by Savannah 
> admins.
> If you have problem with those, you should contact address@hidden .

Right. Thank you.

Your input is truly helpful.

Warm regards.

david

> 
> 
> Hope this helps.
> regards,
> - assaf
>