librsync
2.3.2
|
Manage librsync streams of IO. More...
Go to the source code of this file.
Functions | |
size_t | rs_buffers_copy (rs_buffers_t *stream, size_t len) |
rs_result | rs_tube_catchup (rs_job_t *job) |
Put whatever will fit from the tube into the output of the stream. More... | |
int | rs_tube_is_idle (rs_job_t const *job) |
void | rs_tube_write (rs_job_t *job, void const *buf, size_t len) |
Push some data into the tube for storage. More... | |
void | rs_tube_copy (rs_job_t *job, size_t len) |
Queue up a request to copy through len bytes from the input to the output of the stream. More... | |
void | rs_scoop_input (rs_job_t *job, size_t len) |
Try to accept a from the input buffer to get LEN bytes in the scoop. More... | |
void | rs_scoop_advance (rs_job_t *job, size_t len) |
Advance the input cursor forward len bytes. More... | |
rs_result | rs_scoop_readahead (rs_job_t *job, size_t len, void **ptr) |
Read from scoop without advancing. More... | |
rs_result | rs_scoop_read (rs_job_t *job, size_t len, void **ptr) |
Read LEN bytes if possible, and remove them from the input scoop. More... | |
rs_result | rs_scoop_read_rest (rs_job_t *job, size_t *len, void **ptr) |
Read whatever data remains in the input stream. More... | |
size_t | rs_scoop_total_avail (rs_job_t *job) |
Return the total number of bytes available including the scoop and input buffer. More... | |
Manage librsync streams of IO.
See
OK, so I'll admit IO here is a little complex. The most important player here is the stream, which is an object for managing filter operations. It has both input and output sides, both of which is just a (pointer,len) pair into a buffer provided by the client. The code controlling the stream handles however much data it wants, and the client provides or accepts however much is convenient.
At the same time as being friendly to the client, we also try to be very friendly to the internal code. It wants to be able to ask for arbitrary amounts of input or output and get it without having to keep track of partial completion. So there are functions which either complete, or queue whatever was not sent and return RS_BLOCKED.
The output buffer is a little more clever than simply a data buffer. Instead it knows that we can send either literal data, or data copied through from the input of the stream.
In buf.c you will find functions that then map buffers onto stdio files.
So on return from an encoding function, either the input or the output or possibly both will have no more bytes available.
librsync never does IO or memory allocation, but relies on the caller. This is very nice for integration, but means that we have to be fairly flexible as to when we can `read' or `write' stuff internally.
librsync basically does two types of IO. It reads network integers of various lengths which encode command and control information such as versions and signatures. It also does bulk data transfer.
IO of network integers is internally buffered, because higher levels of the code need to see them transmitted atomically: it's no good to read half of a uint32. So there is a small and fixed length internal buffer which accumulates these. Unlike previous versions of the library, we don't require that the caller hold the start until the whole thing has arrived, which guarantees that we can always make progress.
On each call into a stream iterator, it should begin by trying to flush output. This may well use up all the remaining stream space, in which case nothing else can be done.
Definition in file stream.h.
void rs_tube_write | ( | rs_job_t * | job, |
const void * | buf, | ||
size_t | len | ||
) |
void rs_tube_copy | ( | rs_job_t * | job, |
size_t | len | ||
) |
Queue up a request to copy through len
bytes from the input to the output of the stream.
The data is copied from the scoop (if there is anything there) or from the input, on the next call to rs_tube_write().
We can only accept this request if there is no copy command already pending.
void rs_scoop_input | ( | rs_job_t * | job, |
size_t | len | ||
) |
void rs_scoop_advance | ( | rs_job_t * | job, |
size_t | len | ||
) |
Advance the input cursor forward len
bytes.
This is used after doing readahead, when you decide you want to keep it. len
must be no more than the amount of available data, so you can't cheat.
So when creating a delta, we require one block of readahead. But after examining that block, we might decide to advance over all of it (if there is a match), or just one byte (if not).
Read from scoop without advancing.
Ask for LEN bytes of input from the stream. If that much data is available, then return a pointer to it in PTR, advance the stream input pointer over the data, and return RS_DONE. If there's not enough data, then accept whatever is there into a buffer, advance over it, and return RS_BLOCKED.
The data is not actually removed from the input, so this function lets you do readahead. If you want to keep any of the data, you should also call rs_scoop_advance() to skip over it.
Read LEN bytes if possible, and remove them from the input scoop.
*job | An rs_job_t pointer to the job instance. |
len | The length of the data in the ptr buffer. |
**ptr | will be updated to point to a read-only buffer holding the data, if enough is available. |
Read whatever data remains in the input stream.
*job | The rs_job_t instance the job instance. |
*len | will be updated to the length of the available data. |
**ptr | will point at the available data. |