Apache Mesos
Classes | Functions
recordio Namespace Reference

Provides facilities for "Record-IO" encoding of data. More...

Classes

class  Decoder
 Decodes records from "Record-IO" data (see above). More...
 

Functions

std::string encode (const std::string &record)
 Returns the "Record-IO" encoded record. More...
 

Detailed Description

Provides facilities for "Record-IO" encoding of data.

"Record-IO" encoding allows one to encode a sequence of variable-length records by prefixing each record with its size in bytes:

5
hello 6
world!

Note that this currently only supports record lengths encoded as base 10 integer values with newlines as a delimiter. This is to provide better language portability portability: parsing a base 10 integer is simple. Most other "Record-IO" implementations use a fixed-size header of 4 bytes to directly encode an unsigned 32 bit length.

TODO(bmahler): Make the encoder and decoder zero-copy, once they're just dealing with bytes. To make the encoder zero-copy, we need to make "writes" directly to an output (e.g. call a callback with bytes to write, or write to a provided "output stream" abstraction). For the decoder, we can provide a string view into the input data of the record.

Function Documentation

std::string recordio::encode ( const std::string &  record)
inline

Returns the "Record-IO" encoded record.

Unlike the decoder, this can just be a stateless function since we're taking entire records and each encoded record is independent.