Combining all FEniCS projects into one shared repository
We talked about combining all FEniCS projects into one shared repository. What are the pros and cons? It's a rather big operation, so it's best to be sure. Here's my take, please add other considerations.
I've copied a script below on the whiteboad which builds this shared repository from current trunks,
and the result can be browsed right now on lp:~martinal/+junk/unified-fenics-repo
Pros:
- Easy to fetch entire fenics sourecode
- Atomic merges of crosscutting changes
- Atomic updates on buildbot, avoiding the race condition to happens today if you
- Cleaner versioning
- Less release work
- Less confusion among users updating only some of the repos
- Blueprints, bugs, and discussions are often crosscutting and not associated with one single project
- Shared history of all projects (yes, I put that in pros)
- We can still keep the separated software architecture of the current projects
- But we can also choose to move all python modules into one site-packages/
- We can still keep the building and testing separated (but also add "buildall", "testall" eventually)
Cons:
- It's a big download just to get a single project like ufl or instant
- Slower bzr operations when working with just a small project like ufl
- Outstanding merges will be hard(er) and should be done before the repository merge, requiring some coordination
- .bzrignore files must be merged manually (not really a con, just a small boring job)
- The merged list of old tags is a bit messy (for future releases we can just use fenics-3.0 instead of 3.0)
- Backporting fixes will be tricky (worst case manual patching with diffs)
- For some projects it's unclear whether they should be part of FEniCS or standalone
As you see I land pretty strongly on the pro side now. We just need to avoid rushing things so we all agree there are no foreseeable problems, and execute this operation at a time where merges won't be a biggie.
To test how this will work out locally, get this branch or copy the script below and run it in a clean directory. Then watch the shared fenics/ repository be created from project trunks. To see the old history, try e.g.
bzr log -rufc-history -n0 -l7
to see the last few commits of the ufc history. The history of single files is preserved.
I encourage somebody(tm) to try to figure out how to best handle merging of eventual outstanding feature branches into this new repository structure. Worst case is manual patching of files. I, for one, intend to avoid that issue.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- None
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Discussion
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
#!/bin/bash
# Select which projects to fetch here!
#PROJECTS='instant ufc ufl fiat ferari ffc dorsal uflacs'
PROJECTS='instant ufc ufl fiat ferari ffc dorsal uflacs dolfin'
# Create initial repository
mkdir fenics
pushd fenics
bzr init .
echo Welcome to the new and shiny shared repository for all FEniCS projects. > README
bzr add README
bzr commit -m'Welcome to the new and shiny shared repository for all FEniCS projects.'
popd
# For each project, branch it, move contents to subdir, and merge it into shared repo
mkdir projects
for p in ${PROJECTS}; do
# Get fresh branch
pushd projects
bzr branch lp:$p
# Move to subdir
pushd $p
ls -1 > filelist.tmp
sed -i s/filelist.tmp// filelist.tmp
mkdir $p-project
echo bzr add $p-project
bzr add $p-project
echo bzr mv `cat filelist.tmp` $p-project/
bzr mv `cat filelist.tmp` $p-project/
if [[ -f .bzrignore ]]; then
echo bzr mv .bzrignore $p-bzrignore
bzr mv .bzrignore $p-bzrignore
fi
echo bzr commit -m"Move everything to directory "$p"-project"
bzr commit -m"Move everything to directory "$p"-project"
popd
# Out of projects/
popd
# Merge into fenics
pushd fenics
echo bzr merge -r0..-1 ../projects/$p
bzr merge -r0..-1 ../projects/$p
echo bzr commit -m"Initial merge of "$p" into shared repository."
bzr commit -m"Initial merge of "$p" into shared repository."
echo bzr tag $p-history
bzr tag $p-history
popd
done
# Rename foo-project into just foo
pushd fenics
for p in ${PROJECTS}; do
echo bzr mv $p-project $p
bzr mv $p-project $p
echo bzr commit -m"Rename "$p"-project to "$p"."
bzr commit -m"Rename "$p"-project to "$p"."
done
# Show off the result
echo
echo Congratulations on your new and shiny FEniCS repository!
echo
bzr log
echo
echo Now try e.g.
echo bzr log -rufc-history -n0 -l10
echo to see the last 10 commits of the ufc history before The Merge.
echo
echo One last thing needs to be done: the old .bzrignore files are named
echo foo-bzrignore and must be manually rewritten into a single .bzrignore file.
echo
