Accessing the Blob Store

The lowest-level interface in Camlistore is the raw blob store, which provides a mechanism to store and retrieve immutable objects. All other Camlistore functionality is built on this base layer.

Blob store functionality is accessed via camlistore.Connection.blobs, which is a pre-configured instance of camlistore.blobclient.BlobClient.

class camlistore.blobclient.BlobClient(http_session, base_url)

Low-level interface to Camlistore’s blob store interface.

The blob store is the lowest-level Camlistore API and provides only for inserting and retrieving immutable, content-addressed blobs.

All of the functionality of Camlistore builds on this abstraction, but most use-cases are better served by the search interface, which can be accessed via camlistore.Connection.searcher.

Callers should not instantiate this class directly. Instead, call camlistore.connect() to obtain a camlistore.Connection object and access camlistore.Connection.blobs.

blob_exists(blobref)

Determine if a blob exists with the given blobref.

Returns True if the blobref is known to the server, or False if it is not.

To more efficiently test the presence of many blobs at once, it’s better to use get_size_multi(); known blobs will have a size, while unknown blobs will indicate None.

enumerate()

Enumerate all of the blobs on the server, in blobref order.

Returns an iterable over all of the blobs. The underlying server interface returns the resultset in chunks, so beginning iteration will cause one request but continued iteration may cause followup requests to retrieve additional chunks.

Most applications do not need to enumerate all blobs and can instead use the facilities provided by the search interface. The enumeration interface exists primarily to enable the Camlistore indexer to build its search index, but may be useful for other alternative index implementations.

get(blobref)

Get the data for a blob, given its blobref.

Returns a camlistore.Blob instance describing the blob, or raises camlistore.exceptions.NotFoundError if the given blobref is not known to the server.

get_size(blobref)

Get the size of a blob, given its blobref.

Returns the size of the blob as an int in bytes, or raises camlistore.exceptions.NotFoundError if the given blobref is not known to the server.

get_size_multi(*blobrefs)

Get the size of several blobs at once, given their blobrefs.

This is a batch version of get_size(), returning a mapping object whose keys are the request blobrefs and whose values are either the size of each corresponding blob or None if the blobref is not known to the server.

put(blob)

Write a single blob into the store.

The blob must be given as a camlistore.Blob instance. Returns the blobref of the created blob, which is guaranteed to match blob.blobref of the given blob.

This function will first check with the server to see if it has the given blob, so it is not necessary for the caller to check for the existence of the blob before uploading.

When writing many blobs at once – a more common occurence than just one in most applications – it is more efficient to use put_multi(), since it is able to batch-upload blobs and reduce the number of round-trips required to complete the operation.

put_multi(*blobs)

Upload several blobs to the store.

This is a batch version of put(), uploading several blobs at once and returning a list of their blobrefs in the same order as they were provided in the arguments.

At present this method does not correctly handle the protocol restriction that only 32MB of data can be uploaded at once, so this function will fail if that limit is exceeded. It is intended that this will be fixed in a future version.

class camlistore.Blob(data, hash_func_name='sha1', blobref=None)

Represents a blob.

A blob is really just a raw string of bytes, but this class exists to provide a convenient interface to make a blob and find its blobref and size.

Although blobs are not mutable, instances of this class are. Mutating instances of this class (by assigning to Blob.data or Blob.hash_func_name) will change the blob’s blobref, causing it to be a different blob as far as Camlistore is concerned, although it remains the same object as far as Python is concerned.

Most callers should not pass a blobref argument to the initializer, since it can be computed automatically from the other arguments. If one is provided, it must match the provided data or else the camlistore.exceptions.HashMismatchError exception will be raised, allowing callers to check for a hash mismatch as a side-effect. If a blobref is provided, its hash function overrides the value passed in as hash_func_name.

blobref

The blobref of this blob.

This value will change each time either data or hash_func_name is modified, so callers should be careful about caching this value in a local variable if modifications are expected.

data

The raw blob data, as a str.

Assigning to this property will change blobref, and effectively create a new blob as far as the server is concerned.

hash_func_name

The name of the hash function to use for this blob’s blobref.

This must always be the name of a function that is supported by both the local hashlib and the Camlistore server. "sha1" is currently a safe choice for compatibility, and is thus the default. "sha256" will also work with the implementations available at the time of writing.

Assigning to this property will change blobref, and effectively create a new blob as far as the server is concerned.

size

The size of the blob data, in bytes.

class camlistore.blobclient.BlobMeta(blobref, size=None, blob_client=None)

Metadata about a blob.

This is essentially a camlistore.Blob object without the blob’s data, for situations where we have the identity of a blob but have not yet retrieved it.

Callers should not instantiate this class directly. It’s intended only to be used as the return value of methods on BlobClient.

get_data()

Retrieve the blob described by this object.

This will call to the server to obtain the given blob, with the same behavior as BlobClient.get().