7 * Fix #198 cmake popt detection
using pkg-config and #199 test scripts on
8 FreeBSD. Fixes and tidies FindPOPT.cmake and Findlibb2.cmake to use
9 pkg-config correctly and behave more like official FindPackage() cmake
10 modules. Makes all test scripts use /bin/sh instead of /bin/bash. (dbaarda,
13 * Change default block_len to always be a multiple of the blake2b 128 byte
14 blocksize for efficiency. Tidy and update docs to explain using
16 document new magic types. Remove really obsolete entries in TODO.md. Update
17 to Doxygen 1.8.16. (dbaarda, https:
19 * Improve
hashtable performance by adding a small optional bloom filter,
20 reducing max loadfactor from 80% to 70%, Fix hashcmp_count stats to include
21 comparing against empty buckets. This speeds up deltas by 20%~50%.
26 * Optimize rabinkarp_update() by correctly using
unsigned constants and
27 manually unrolling the loop for best performance. (dbaarda,
34 * Bump minor version from 2.2.1 to 2.3.0 to reflect additional
rs_sig_args()
35 and strong_len=-1 support.
37 * Add public
rs_sig_args() function for getting the recommend signature args
38 from the file size. Added support to rdiff for `--sum-size=-1` to indicate
39 "use minimum size safe against random block collisions". Added warning
40 output for sum-sizes that are too small to be safe. Fixed possible rdiff
41 bug affecting popt parsing on non-little-endian platforms. (dbaarda,
44 * Fixed yet more compiler warnings for various platforms/compilers.
45 (Adsun701, texierp, https:
48 * Improved cmake popt handling to find popt dependencies using PkgConfig.
51 * Tidied internal code and improved tests for netint.[ch], tube.c, and
55 * Improved C99 compatibility. Add `-std=c99 -pedantic` to `CMAKE_C_FLAGS` for
56 gcc and clang. Fix all C99 warnings by making all code C99 compliant. Tidy
57 all CMake checks, `
#cmakedefines`, and `#includes`. Fix 64bit support for
58 mdfour checksums (texierp, dbaarda,
62 * Usage clarified in rdiff (1) man page. (AaronM04,
69 * Fix #176 hangs calculating deltas
for files larger than 4GB. (dbaarda,
76 * Bump minor version from 2.1.0 to 2.2.0 to reflect additional RabinKarp
79 * Fix MSVC builds by adding missing LIBRSYNC_EXPORT to variables in
80 librsync.h, add -DLIBRSYNC_STATIC_DEFINE to the sumset_test target,
81 and correctly install .dll files in the bin directory.
84 * Add RabinKarp rollsum support and make it the
default. RabinKarp is a much
85 better rolling hash, which reduces the risk of hash collision corruption
86 and speeds up delta calculations. The rdiff cmd gets a
new `-R
87 (rollsum|rabinkarp)` argument with the
default being `rabinkarp`, Use `-R
88 rollsum` to generate backwards-compatible signatures. (dbaarda,
91 * Use single-byte literal commands
for small inserts in deltas. This makes
92 each small insert use 1 less byte in deltas. (dbaarda,
95 * Fix multiple warnings (cross-)compiling
for windows. (Adsun701,
99 * Change
rs_file_size() to report -1 instead of 0
for unknown file sizes (not
100 a regular file). (dbaarda https:
102 * Add cmake BUILD_SHARED_LIBS option
for static library support.
103 BUILD_SHARED_LIBS defaults to ON, and can be set to OFF
using `ccmake .` to
104 build librsync as a
static library. (dbaarda
107 * Fix compile errors and add .gitignore entries
for MSVS 2019. Fixes
115 * Bump minor version from 2.0.3 to 2.1.0 to reflect additions to librsync.h.
117 * Fix exporting of
private symbols from librsync library. Add export of
120 log signature
hashtable hit/miss stats. Improve rdiff error output.
123 * Updated release process to include stable tarballs. (dbaarda,
126 * Remove redundant and broken `--paranoia` argument from rdiff. (dbaarda,
130 (telles-simbiose, https:
132 * Document delta file format. (zmj,
135 * Fix up doxygen comments. (dbaarda,
142 * Improve CMake install paths configuration (wRAR,
144 checking when cross-compiling (fornwall,
147 * Fix Unaligned memory access
for rs_block_sig_init() (dbaarda,
150 * Fix hashtable_test.c name collision
for key_t in sys/types.h on some
151 platforms (dbaarda, https:
153 * Format code with consistent style, adding `make tidy` and `make
154 tidyc` targets
for reformating code and comments. (dbaarda,
157 * Removed perl as a build dependency. Note it is still required
for some
158 tests. (dbaarda, https:
160 * Update RPM spec file
for v2.0.2 and fix cmake man page install. (deajan,
167 * Extensively reworked Doxygen documentation, now available at
170 * Removed some declarations from librsync.h that were unimplemented or no
171 longer ever useful: `rs_work_options`, `rs_accum_value`. Remove
172 declaration of unimplemented `rs_mdfour_file()`. (Martin Pool)
174 * Remove shipped `snprintf` code: no longer acutally linked after changing to
175 CMake, and since it
's part of C99 it should be widely available.
178 * Document that Ninja (http://ninja-build.org/) is supported under CMake.
179 It's a bit faster and nicer than Make. (Martin Pool)
181 * `make check` (or `ninja check` etc) will now build and run the tests.
182 Previously due to a CMake limitation, `make test` would only run existing
183 tests and could fail
if they weren
't built.
184 (Martin Pool, https://github.com/librsync/librsync/issues/49)
186 * Added cmake options to exclude rdiff target and compression from build.
187 See install documentation for details. Thanks to Michele Bertasi.
189 * `popt` is only needed when `rdiff` is being built. (gulikoza)
191 * Improved large file support for platforms using different variants
192 of `fseek` (`fseeko`, `fseeko64`, `_fseeki64`), `fstat` (`fstat64`,
193 `_fstati64`), and `fileno` (`_fileno`). (dbaarda, charlievieth,
194 gulikoza, marius-nicolae)
196 * `rdiff -s` option now shows bytes read/written and speed. (gulikoza).
197 For delta operations it also shows hashtable match statistics. (dbaarda)
199 * Running rdiff should not overwrite existing files (signatures, deltas and
200 new patched files) by default. If the destination file exists, rdiff will
201 now exit with an error. Add new option -f (--force) to overwrite existing
204 * Improve signature memory allocation (doubling size instead of calling
205 realloc for every sig block) and added support for preallocation. See
206 streaming.md job->estimated_signature_count for usage when using the
207 library. `rdiff` uses this by default if possible. (gulikoza, dbaarda)
209 * Significantly tidied signature handling code and testing, resulting in more
210 consistent error handling behaviour, and making it easier to plug in
211 alternative weak and strong sum implementations. Also fixed "slack delta"
212 support for delta calculation with no signature. (dbaarda)
214 * `stdint.h` and `inttypes.h` from C99 is now required. Removed redundant
215 librsync-config.h header file. (dbaarda)
217 * Lots of small fixes for windows platforms and building with MSVC.
218 (lasalvavida, mbrt, dbaarda)
220 * New open addressing hashtable implementation that significantly speeds up
221 delta operations, particularly for large files. Also fixed degenerate
222 behaviour with large number of duplicate blocks like runs of zeros
223 in sparse files. (dbaarda)
225 * Optional support with cmake option for using libb2 blake2 implementation.
226 Also updated included reference blake2 implementation with bug fixes
229 * Improved default values for input and output buffer sizes. The defaults are
230 now --input-size=0 and --output-size=0, which will choose recommended
231 default sizes based on the --block-size and the operation being performed.
234 * Fixed hanging for truncated input files. It will now correctly report an
235 error indicating an unexpected EOF was encountered. (dbaarda,
236 https://github.com/librsync/librsync/issues/32)
238 * Fixed #13 so that faster slack delta's are used
for signatures of
239 empty files. (dbaarda,
242 * Fixed #33 so
rs_job_iter() doesn
't need calling twice with eof=1.
243 Also tidied and optimized it a bit. (dbaarda,
244 https://github.com/librsync/librsync/issues/33)
246 * Fixed #55 remove excessive rs_fatal() calls, replacing checks for
247 programming errors with assert statements. Now rs_fatal() will only
248 be called for rare unrecoverable fatal errors like malloc failures or
249 impossibly large inputs. (dbaarda,
250 https://github.com/librsync/librsync/issues/55)
256 Note: despite the major version bump, this release has few changes and should
257 be binary and API compatible with the previous version.
259 * Bump librsync version number to 2.0, to match the library
260 soname/dylib version.
261 (Martin Pool, https://github.com/librsync/librsync/issues/48)
263 ## librsync 1.0.1 (2015-11-21)
265 * Better performance on large files. (VictorDenisov)
267 * Add comment on usage of rs_build_hash_table(), and assert correct use.
268 Callers must call rs_build_hash_table() after loading the signature,
269 and before calling rs_delta_begin().
270 Thanks to Paul Harris <paulharris@computer.org>
272 * Switch from autoconf to CMake.
274 Thanks to Adam Schubert.
276 ## librsync 1.0.0 (2015-01-23)
278 * SECURITY: CVE-2014-8242: librsync previously used a truncated MD4
279 "strong" check sum to match blocks. However, MD4 is not cryptographically
280 strong. It's possible that an attacker who can control the contents of one
281 part of a file could use it to control other regions of the file,
if it
's
282 transferred using librsync/rdiff. For example this might occur in a
283 database, mailbox, or VM image containing some attacker-controlled data.
285 To mitigate this issue, signatures will by default be computed with a
286 256-bit BLAKE2 hash. Old versions of librsync will complain about a
287 bad magic number when given these signature files.
289 Backward compatibility can be obtained using the new
290 `rdiff sig --hash=md4`
291 option or through specifying the "signature magic" in the API, but
292 this should not be used when either the old or new file contain
295 Deltas generated from those signatures will also use BLAKE2 during
296 generation, but produce output that can be read by old versions.
298 See https://github.com/librsync/librsync/issues/5
300 Thanks to Michael Samuel <miknet.net> for reporting this and offering an
303 * Various build fixes, thanks Timothy Gu.
305 * Improved rdiff man page from Debian.
307 * Improved librsync.spec file for building RPMs.
309 * Fixed bug #1110812 'internal error: job made no progress
'; on large
312 * Moved hosting to https://github.com/librsync/librsync/
314 * Travis-CI.org integration test at https://travis-ci.org/librsync/librsync/
316 * You can set `$LIBTOOLIZE` before running `autogen.sh`, for example on
317 OS X Homebrew where it is called `glibtoolize`.
319 ## 0.9.7 (released 2004-10-10)
321 * Yet more large file support fixes.
323 * `extern "C"` guards in librsync.h to let it be used from C++.
325 * Removed Debian files from dist tarball.
327 * Changed rdiff to an installed program on "make install".
329 * Refactored delta calculation code to be cleaner and faster.
331 * \#879763: Fixed mdfour to work on little-endian machines which don't
332 like unaligned word access. This should make librsync work on
333 pa-risc, and it makes it slightly faster on ia64.
335 * \#1022764: Fix corrupted encoding of some COPY commands in large
338 * \#1024881: Print
long integers directly, rather than via casts to
341 * Fix printf formats
for size_t: both the format and the argument
342 should be cast to
long.
346 * Large file support fixes.
348 * [v]snprintf or _[v]snprintf autoconf replacement
function fix.
350 * Changed installed include file from rsync.h to librsync.h.
352 * Migration to sourceforge
for hosting.
354 * Rollsum bugfix that produces much smaller deltas.
356 * Memory leaks bugfix patches.
358 * mdfour bigendian and >512M bugfix, plus optimisations patch.
360 * autoconf/automake updates and cleanups
for autoconf 2.53.
362 * Windows compilation patch, heavily modified.
364 * MacOSX compilation patch, modified to autoconf vararg macro fix.
366 * Debian package build scripts patch.
370 * Bugfix patch from Shirish Hemant Phatak
372 ## 0.9.4: (library 1.1.0)
374 * Fixes
for rsync.h from Thorsten Schuett <thorsten.schuett@zib.de>
376 * RLL encoding fix from Shirish Hemant Phatak <shirish@nustorage.com>
378 * RPM spec file by Peter J. Braam <braam@clusterfs.com>
380 * No (intentional) changes to binary API.
384 * Big speed improvements in MD4 routines and generation of weak
387 * Patch to build on FreeBSD by Jos Backus <josb@cncdsl.com>
389 * Suggestions to build on Solaris 2.6 from Alberto Accomazzi
390 <aaccomazzi@cfa.harvard.edu>
392 * Add
rs_job_drive, a
generic mechanism
for turning the library into
393 blocking mode. rs_whole_run now builds on top of
this. The
394 filebuf interface has changed a little to accomodate it.
396 * Generating and loading signatures now generates statistics.
400 * I suspect there may be a bug in rolling checksums, but it probably
401 only causes inefficiency and not corruption.
403 * Portability fixes
for alphaev67-dec-osf5.1; at the moment builds
404 but does not work because librsync tries to
do unaligned accesses.
406 * Works on sparc64-unknown-linux-gnu (Debian/2.2)
410 * Improve delta algorithm so that deltas are actually
411 delta-compressed, rather than faked.
415 * Rename the library to `librsync
'.
419 * Include the popt library, and use it to build rdiff if the library
420 is not present on the host.
422 * Add file(1) magic for rdiff.
424 * Add more to the manual pages.
426 * It's no longer necessary to call rs_buffers_init on a stream before
427 starting to use it: all the
internal data is kept in the job, not
430 * Rename rs_stream_t to
rs_buffers_t, a more obvious name. Pass the
431 buffers to every
rs_job_iter() call, rather than setting it at
432 startup. Similarly
for all the _begin() functions.
434 * rs_job_new also takes the initial state
function.
440 * Redesign API to be more like zlib/bzlib.
442 * Put all command-line functions into a single rdiff(1) program.
444 * New magic number `rs6
'
446 * Change to using popt for command line parsing.
448 * Use Doxygen for API documentation.
452 * Changes stats string format.
454 * Slightly improved test cases
458 * Don't install debugging tools into /usr/local/bin; leave them in
459 the source directory.
461 * Fix libhsync to build on (sgi-mips, IRIX64, gcc, GNU Make)
463 * Include README.CVS in tarball
465 * Back out of
using libtool and shared libraries, as it is
466 unnecessary at
this stage, complicates installation and slows down
469 * Use mapptr when reading data to decode, so that decoding should
470 have less latency and be more reliable.
472 * Cope better on systems that are missing functions like snprintf.
476 * Put genuine search encoding back into the nad algorithm, and
477 further clean up the nad code. Literals are now sent out
using a
478 literal buffer integrated with the input mapptr so that data is not
479 copied. Checksums are still calculated from scratch each time
480 rather than by rolling --
this is very slow but simple.
482 * Reshuffle test cases so that they use files generated by hsmapread,
483 rather than the source directory. This makes the tests quicker and
484 more reproducible, hopefully without losing coverage. Further
485 develop the test driver framework.
487 * Add hsdumpsums debugging tool.
489 * Hex strings (eg strong checksums) are broken up by underscores
for
492 * Stats now go to the log rather than stdout.
494 * mapptr acts properly when we
're skipping/rewinding to data already
495 present in the buffer -- it does a copy if required, but not
500 * Improved mapptr input code
502 * Turn on more warnings if using gcc
508 * Improvements to mapptr to make it work better for network IO.
510 * Debug trace code is compiled in unless turned off in ./configure
511 (although most programs will not write it out unless asked.)
513 * Add libhsyncinfo program to show compiled-in settings and version.
515 * Add test cases that run across localhost TCP sockets.
517 * Improved build code; should now build easily from CVS through
520 * Improved trace code.
522 * Clean up to build on sparc-sun-solaris2.8, and in the process clean
523 up the handling of bytes vs chars, and of building without gcc
525 * Reverse build scripts so that driver.sh calls the particular
530 * Use mapptr for input.
532 * Implement a new structure for encoding in nad.c. It doesn't
533 encode at the moment, but it
's much more maintainable.
535 * More regression cases.
537 * Clean up build process.
541 * Rewrite hs_inbuf and hs_encode to make them simpler and more
544 * Test cases for input handling.
546 * Use the map_ptr idea for input from both streams and files.
550 * automake/autoconf now works cleanly when the build directory is
551 different to the source directory.
553 * --enable-ccmalloc works again.
557 * A much better regression suite.
559 * CHECKSUM token includes the file's checksum up to the current
560 location, to aid in
self-testing.
562 * Various bug fixes, particularly to
do with
short IO returns.
Description of input and output buffers.
rs_result rs_job_iter(rs_job_t *job, rs_buffers_t *buffers)
Run a rs_job state machine until it blocks (RS_BLOCKED), returns an error, or completes (RS_DONE)...
Bad value passed in to library, probably an application bug.
void * block_sigs
The packed block_sigs for all blocks.
LIBRSYNC_EXPORT int rs_file_close(FILE *file)
Close a file with special handling for stdin or stdout.
Signature of a whole file.
LIBRSYNC_EXPORT rs_result rs_build_hash_table(rs_signature_t *sums)
Call this after loading a signature to index it.
LIBRSYNC_EXPORT FILE * rs_file_open(char const *filename, char const *mode, int force)
Open a file with special handling for stdin or stdout.
LIBRSYNC_EXPORT void rs_signature_log_stats(rs_signature_t const *sig)
Log the rs_signature_delta match stats.
LIBRSYNC_EXPORT rs_result rs_sig_args(rs_long_t old_fsize, rs_magic_number *magic, size_t *block_len, size_t *strong_len)
Get or check signature arguments for a given file size.
rs_result rs_job_drive(rs_job_t *job, rs_buffers_t *buf, rs_driven_cb in_cb, void *in_opaque, rs_driven_cb out_cb, void *out_opaque)
Actively process a job, by making callbacks to fill and empty the buffers until the job is done...
LIBRSYNC_EXPORT rs_long_t rs_file_size(FILE *file)
Get the size of a file.