monotone

monotone Mtn Source Tree

Root/UPGRADE

1upgrading monotone to 0.40
2==========================
3
4How to read this file:
5 - Figure out what version of monotone you are upgrading from, let's
6 say 0.22, for example.
7 - Read down the list, and at each "version X or earlier" line, those
8 instructions apply if X is larger than 0.22. So, for instance,
9 the "0.25 or earlier" instructions _and_ the "0.23 or earlier"
10 instructions both apply.
11 - If there are no entries that apply, then there is nothing to do.
12 For instance, when upgrading from 0.23 to 0.24, no action was
13 necessary.
14 - No matter what, reading the release notes in NEWS is a good idea.
15 Changes to things like command line switches, new commands, etc.,
16 will be described there, not here.
17
18If you are upgrading from:
19 - 0.39 or earlier: The database format has changed. You must run
20 (after remembering to take a backup copy):
21 $ mtn -d mydb.mtn db migrate
22 on each of your databases.
23 - 0.31 or earlier: The database format has changed. You must run
24 (after remembering to take a backup copy):
25 $ mtn -d mydb.mtn db migrate
26 $ mtn -d mydb.mtn db regenerate_caches
27 on each of your databases.
28 - 0.29 or earlier: The workspace format has changed. You must run:
29 $ mtn migrate_workspace
30 in each of your workspaces.
31 - 0.25 or earlier: Please read this section carefully. We have
32 modified the textual format used by revisions and manifests; old
33 monotones cannot parse the new format, and new monotones cannot
34 parse the old format, except to convert it to the new format. See
35 NEWS before upgrading. Your project must rebuild its history;
36 this is irreversible and creates a flag day -- everyone in your
37 project must switch over at the same time.
38
39 You will not be able to sync revisions between pre- and post-
40 migration databases, and each project should only run the
41 migration ONCE, on ONE database. Pick a person to do the
42 migration, they do it, once they're done, everyone else does a
43 fresh pull from them.
44
45 You should probably do some test migrations locally first to make
46 sure everything seems to be working, before running your real
47 migration. To run the real migration:
48 1) get everyone to commit and push their changes to the database
49 chosen for migration (such as a central server if you use
50 one); local dbs and working copies will be broken by this
51 migration.
52 2) shut down the server
53 3) run 'cp server.db server.db.backup'
54 4) run 'mtn db migrate --db server.db'
55 5) run 'mtn db rosterify --db server.db'
56 6) do whatever tests you like to make sure things look reasonable
57 7) make sure that your server's firewall is set up to allow
58 connections on the new netsync port: 4691
59 8) start up the server
60 9) tell everyone to do a new pull into a fresh database, using
61 the new version of monotone.
62 This migration is "best effort"; bugs in rename handling in the
63 old code mean that some histories contain bogosities that cannot
64 be migrated. However, it makes an effort to preserve everything
65 it can (including renames), and does guarantee that at each
66 entry in the history graph, the tree layout and file contents are
67 preserved exactly.
68
69 The largest changes are that after upgrading, all revision ids
70 will change, and, as a consequence, all certs must be reissued.
71 This means that after upgrading, all certs will be signed by the
72 person who performed the upgrade, rather than their original
73 issuer. This is a basic property of how signatures work, so
74 there's little we can do about it.
75
76 If you have any private branches, and are not your project's
77 "designated migrator", then it is still possible to preserve your
78 branches. However, this requires some more understanding of what
79 exactly is going on, and is outside the scope of this document.
80 See http://venge.net/mtn-wiki/RosterifyingPrivateBranches
81 for one approach, and if you don't understand the logic behind
82 those commands, then we're happy to explain things in more detail
83 on the mailing list (monotone-devel@nongnu.org) or IRC (#monotone
84 on irc.OFTC.net).
85
86 We tentatively hope that this is the last time we will have to
87 change the revision/manifest formats and have a flag day of this
88 magnitude; but only experience will let us know for certain.
89
90 If you have any questions or concerns about this process, please
91 let us know via email (monotone-devel@nongnu.org) or discuss it on
92 IRC (#monotone on irc.OFTC.net).
93
94 - 0.23 or earlier: keys are now stored in ~/.monotone/keys (Unix,
95 OS X), or %APPDATA%\Monotone\keys (Windows). You must run 'db
96 migrate' against each of your databases; this will automatically
97 migrate the keys. See NEWS for details.
98 Command line syntax for 'serve' has changed; please adjust startup
99 scripts accordingly.
100 On Windows only, monotone now looks in a different place for the
101 monotonerc file; see NEWS for 0.24 for details.
102
103 - 0.21 or earlier: hooks governing netsync read permission have
104 changed again; see NEWS for 0.22.
105
106 - 0.19 or earlier: there are incompatible command line and server
107 configuration changes; see NEWS for 0.20.
108
109 - 0.18 or earlier: if you have created a ~/.monotonerc, rename it to
110 ~/.monotone/monotonerc, so monotone will still find it.
111
112 - 0.17: simply make a backup of your databases, just in case, and
113 run "db migrate" on each.
114
115 - 0.15 or 0.16: see below
116 - 0.14 or earlier: see file README.changesets
117
118upgrading from 0.15 or 0.16
119---------------------------
120
121there was still some residual badness in the revision graph code in
1220.16. we think we've caught it all now (we hope!), and there is now a
123great deal more coordinated effort put into stopping any such thing
124from sneaking in again, but... we have to do something about the
125badness that's already there.
126
127the solution is, another rebuild, like we did for the 0.15 -> 0.16
128transition. this is still obnoxious, still loses a little bit of
129history information (every revision remains exactly reconstructable,
130but rename information and information on who signed which certs are
131both lost), and still requires coordination among your whole team to
132pull of smoothly. we're sorry. we'll try not to do this again.
133
134just in case we _do_ have to do it again, though, and to help make
135this time smoother, we've added support for "epochs". there's a whole
136new section in the manual about all this, but basically, epochs are a
137way to let monotone keep track of who's done a rebuild, so you can't
138accidentally mix together pre-rebuild and post-rebuild databases.
139(epochs will also come in handy when it's time to migrate away from
140SHA1, for instance.) this means you have more of a safety net than
141last time, though you still have to coordinate within your team...
142
143so, the basic procedure is: one designated person gets to perform the
144rebuild, and deal with any missing files (see below). everyone else
145gets to 1) make sure the designated person has sync'ed with everyone,
146because anything that's not in the designated person's database will
147be lost, or at least, hard to deal with, 2) take a break, so the
148designated person can rebuild and test and such without having to deal
149with new commits.
150
151if you're the designated person, then you should:
152 1) make sure you have everyone's changes, and that they know they
153 shouldn't make any more changes until you give the all-clear
154 2) make a backup copy of your database. seriously, do this.
155 $ cp my.db my.db-backup
156 if something goes wrong, and you don't have a backup, there may
157 not be much we can do to help...
158 3) dump your database to SQL text using your old version of
159 monotone, and then reload it using your 0.17 version of
160 monotone. (this will migrate you from SQLite 2 to SQLite 3,
161 which have different on-disk formats.)
162 $ old-monotone --db=my.db db dump > my-db.dumped
163 $ monotone --db=new.db db load < my-db.dumped
164 (if you've been tracking the 0.17 development mainline, you can
165 skip this step. you still need to do all the others.)
166 4) do the usual 'db migrate' command, for migrating a database to a
167 later version:
168 $ monotone --db=new.db db migrate
169 (this will create the new tables needed for epoch support)
170 5) rebuild your ancestry. as mentioned above, this will lose
171 renames and cert signing information (all certs that you trust
172 will be re-issued with your signature; all other certs will be
173 lost), but will also generate new changesets that actually make
174 sense.
175 $ monotone --db=new.db db rebuild
176 6) check your database. this is the rough equivalent of running a
177 'fsck' or a 'scandisk' on your hard drive -- it just goes through
178 and makes sure that everything looks good. it's especially
179 important for detecting missing files; see below.
180 $ monotone --db=new.db db check 2>&1 | tee db-check.log
181 7) look at db-check.log, and if any files were missing, note down
182 their SHA1's, find the files, and load them into the database
183 with "fload".
184 $ monotone --db=new.db fload <missing-file
185 after doing this, run 'db check' again, until you get a clean
186 bill of health. (if you see problems other than missing files
187 and incomplete manifests/incomplete revisions, then there's
188 something more gone wrong, and you might want to ask the list at
189 monotone-devel@nongnu.org.)
190 8) put the new database somewhere where people can netsync to it.
191 (either by putting it there directly, or by creating a new
192 database on your server and pushing to it.)
193 9) tell everyone to create a new, empty database, and pull from your
194 server. (make sure they remember to copy their private keys over
195 from their old database, using 'monotone privkey' and 'monotone
196 read'.)
197
198missing files
199-------------
200
201unfortunately, monotone 0.16 had a bug that may have led to loss of
202data under certain, rare, circumstances. the bug is that, 0.16's
203"rebuild" (and probably "changesetify") commands incorrectly
204constructed root revisions. as a result, if you had any files that
205were in the initial import of your project, and then later deleted,
206without having had any changes made in between, then those files were
207invisible to netsync, and not transferred by "push", "pull", or
208"sync". therefore, if you rebuilt your database at the 0.16 release,
209and then pulled everything into a new database and deleted the
210original database, monotone has lost its record of those files's
211contents; if you attempted to check out a revision containing them you
212would get an assertion error.
213
214hopefully this should be a rare enough problem that most people don't
215run into it. the new 'db check' command detects this problem (indeed,
216it's how it was discovered in the first place), so it should be easy
217to find out whether you have it or not. if you _do_ have it, you need
218to find those missing files, and tell monotone about them. this is
219straightforward, though it may be tricky -- basically, 'db check' will
220tell you the SHA1's of the missing data. you need to find files that
221have those SHA1's, and load them into your master database. (that
222would be the database that you just ran 'db rebuild', as above.)
223
224some places to look:
225 -- old releases of your software
226 -- old copies of your monotone database (e.g., if you still have
227 pre-changesetify backups of it)
228 'cat file <SHA1>' is the useful command here
229 -- the database that you originally ran 'rebuild' or 'changesetify'
230 in, back at the 0.16 release. since the problem only affects
231 netsync, your initial database should be fine
232
233getting the manifest of your initial revision may also be helpful,
234since you can grep it to find the filenames of the missing files. you
235can do this by examining the output of 'cat revision <revision id>',
236and then running 'cat manifest <manifest id>'.
237
238once you've found the files, load them into your database with
239'fload', e.g.:
240 $ monotone --db=new.db fload <a-found-file
241
242if you cannot find the relevant files anywhere, then there's a more
243serious problem; your history is not fully reconstructible, and you
244won't be able to use it with 0.17, because the bug is now fixed. it
245should still be possible to reconstruct all revisions after the
246offending files were deleted, but this requires some code that doesn't
247currently exist; if you actually are in this state, please email
248<monotone-devel@nongnu.org> and we'll work something out.
249
250in the future, the more rigorous checking monotone now does should
251prevent problems like this from arising; in case you're nervous,
252though, you can always run 'db check' to check for problems.

Archive Download this file

Branches

Tags

Quick Links:     www.monotone.ca    -     Downloads    -     Documentation    -     Wiki    -     Code Forge    -     Build Status