Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
19 check-ins using file tools/cvs2fossil/lib/c2f_file.tcl version 995eee5b4c
|
2007-12-05
| ||
| 02:21 | Removed lots of now dead code. Added a note to the last remaining user of the changeset method 'nextmap'. ... (check-in: 3c0ef2c379 user: aku tags: trunk) | |
|
2007-12-04
| ||
| 04:54 | Reworked ComputeLimits in the last breaker pass. Moved the heavy computation of the max predecessor / min successor data down to the sql in the changeset class. ... (check-in: 711e000206 user: aku tags: trunk) | |
|
2007-12-02
| ||
| 23:47 | Fluff: Renamed state methods use/reading/writing to usedb/use/extend for clarity. Updated all callers. Extended state module with code to dump the SQL statements it receives to a file for analysis. Extended the 'use' declarations of several passes. ... (check-in: e288af3995 user: aku tags: trunk) | |
| 23:45 | Actually put the code for the new pass into the repository. ... (check-in: 060513f2a9 user: aku tags: trunk) | |
| 20:06 | Importer Status... Speed. This is now mostly acceptable. The one exception is pass 'BreakAllCsetCycles'. The reason is that the limit computation it does for backward branches still uses the inefficient file-level dependency computation. This will be tackled in short order. For the other passes the file spent is 'CsetsDeps' is recouped by the much faster graph setup. Memory. This seems to be mostly acceptable as well, with the exceptions of 'BreakAllCsetCycles' (again, for reasons see above), and 'InitCsets'. It seems to happen while the pass breaks internal dependencies, but there is no hard data. I have to measure using a memory-debug enabled tclsh. I suspect either the actual internal dependencies, or the pseudo-dependencies. Maybe combined with a bad choice of data structures. Well, measuring first. ... (check-in: e8c374f670 user: aku tags: trunk) | |
| 20:04 | The performance was still not satisfying, even with faster recomputing of successors. Doing it multiple times (Building the graph in each breaker and sort passes) eats time. Caching in memory blows the memory. Chosen solution: Cache this information in the database. Created a new pass 'CsetDeps' which is run between 'InitCsets' and 'BreakRevCsetCycles' (i.e. changeset creation and first breaker pass). It computes the changeset dependencies from the file-level dependencies once and saves the result in the state, in the new table 'cssuccessor'. Now the breaker and sort passes can get the information quickly, with virtually no effort. The dependencies are recomputed incrementally when a changeset is split by one of the breaker passes, for its fragments and its predecessors. The loop check is now trivial, and integrated into the successor computation, with the heavy lifting for the detailed analysis and reporting moved down into the type-dependent SQL queries. The relevant new method is 'loops'. Now that the loop check is incremental the pass based checks have been removed from the integrity module, and the option '--loopcheck' has been eliminated. For paranoia the graph setup and modification code got its loop check reinstated as an assert, redusing the changeset report code. Renumbered the breaker and sort passes. A number of p... (check-in: 00bf8c198e user: aku tags: trunk) | |
| 06:58 | Added progress output to the breaking of backward branches. ... (check-in: a437da486d user: aku tags: trunk) | |
| 06:57 | Finally committing another disabled integrity constraint for changesets, explanations in the comments. ... (check-in: fd93aa26a6 user: aku tags: trunk) | |
| 06:49 | Bugfix. Typo. ... (check-in: c784751485 user: aku tags: trunk) | |
| 06:18 | Modified loop check in cycle breaker to abort immediately on first looped changeset. ... (check-in: 487bb1c619 user: aku tags: trunk) | |
| 06:17 | Fix table linkage in query, and duplicated conditions :( ... (check-in: f7cca3f082 user: aku tags: trunk) | |
| 05:49 | Performance bugfix. nextmap/premap can still be performance killers and memory hogs. Moved the computation of sucessor changesets down to the type-dependent code (new methods) and the SQL database, i.e. the C level. In the current setup it was possible that the DB would deliver us millions of file-level dependency pairs which the Tcl level would then reduce to tens of actual changeset dependencies. Tcl did not cope well with that amount of data. Now the reduction happens in the query itself. A concrete example was a branch in the Tcl CVS generating nearly 9 million pairs, which reduced to roughly 200 changeset dependencies. This blew the memory out of the water and the converter ground to a halt, busily swapping. Ok, causes behind us, also added another index on 'csitem(iid)' to speed the search for changesets from the revisions, tags, and branches. ... (check-in: 9c57055025 user: aku tags: trunk) | |
| 04:55 | Bugfix. Have the symbol dependency retrieval commands actually return something. ... (check-in: 712010580a user: aku tags: trunk) | |
| 03:49 | Deactivated caching of the nextmap/premap data, with the indices the retrieval seems to be fast enough to allow us to reduce mem consumption. Tweaked log output, and sql formatting. ... (check-in: ac02614803 user: aku tags: trunk) | |
| 03:46 | Bugfix in ValidateFragments, tweaked comment a bit, bugfix in SQL, reordered tables in the successor/predecessor queries a bit to show the actual progression of their use. ... (check-in: fbfb531868 user: aku tags: trunk) | |
| 03:42 | Fixed typo. Grr. ... (check-in: 0a97ad385a user: aku tags: trunk) | |
| 03:41 | Added progress output to the code loading up the graph to traverse, nodes, and arcs. ... (check-in: 41d41c7b57 user: aku tags: trunk) | |
| 03:40 | Added ability to declare indices on tables in the persistent state. Used this to declare indices on critical columns. Slows down the transactions saving changesets, this however is made up when it comes to successor/predecessor retrieval of changesets. ... (check-in: 74854a30b8 user: aku tags: trunk) | |
|
2007-12-01
| ||
| 18:36 | Bugfix. Handle free-floating branches (their root revision was removed as unnecessary). ... (check-in: 79c227a9c0 user: aku tags: trunk) | |