DbFiles implementation

Dbfiles is a library, which provides the service to store files in the database. It uses two database tables, files and files_data. The table files is used to store all the information about the files except the content of the files. The files_data table stores the content of the files as variable length bytea. There are two operation modes, dbfiles can do management of folders or management of folders can be done by the App. This interface makes use of tasks.h. This means all requests start tasks and the UTask callbacks are used to indicate completion.

To create an IDbFiles instance, at first an instance of IDbFilesProvider must be created. This is done by calling CreateDbFilesProvider.

File information

Filecommon/interface/dbfiles.h

Public functions CreateDbFilesProvider

Classes IDbFilesProvider
IDbFiles
IDbFilesStart
IDbFilesPut
IDbFilesGet
IDbFilesDel
IDbFilesMove
IDbFilesList
IDbFilesPathInfo

Data types DBFILES_ROOT
DBFILES_CRYPT_KEY_LEN

Examples Example

Logging

To enable logging, the flag LOG_DBFILES must be set in the managers diagnostic settings.

Encryption

You can enable optional encryption for files data. The used encryption is AES with GCM.
You cannot change the used encryption key after you've already used one! If you change the key, already stored files will become useless and you'll get a TaskFailed callback from the DbFilesGet task.
But you can switch between enabled and disabled encryption, as each file is marked as encrypted or not.

Functions

Functions to initialize

extern "C" IDbFilesProvider * CreateDbFilesProvider();

Overview

This function will create an IDbFilesProvider object which can be used to create IDbFiles instances.

CreateDbFilesProvider

Return value

The IDbFilesProvider instance. This instance can be freed as soon as it is no longer used by calling the C++ delete operator.

Classes

IDbFilesProvider

class IDbFilesProvider {
public:
    class IDbFiles * CreateDbFiles(class IIoMux * iomux, class IInstanceLog * const log, const byte * cryptKey = nullptr);
};

Public functions

CreateDbFiles
This function creates a new IDbFiles object which can be used to create tasks for writing, reading, ... of files.

Parameters

class IIoMux * const iomuxThe IIoMux instance.
class IInstanceLog * const logThe IInstanceLog instance used for loging purposes.
const byte * cryptKeyIf given, files data is stored AES GCM encrypted with this key. The key must be DBFILES_CRYPT_KEY_LEN bytes long!

IDbFiles

class IDbFiles {
public:
    class IDbFilesStart * Start(class UTask * user, class IDatabase * database, const char * folder);
    class IDbFilesPut * Put(class UTask * user, const char * name, ulong64 folder, bool is_folder = false, bool append = false);
    class IDbFilesGet * Get(class UTask * user, ulong64 id, unsigned offset = 0, bool progress = false);
    class IDbFilesDel * Del(class UTask * user, ulong64 id);
    class IDbFilesMove * Move(class UTask * user, ulong64 id, const char * name, ulong64 folder);
    class IDbFilesList * List(class UTask * user, ulong64 folder, ulong64 limit, ulong64 idOffset = 0);
    class IDbFilesPathInfo * PathInfo(class UTask * user, const char * path);
};

Public functions

Start
Creates an IDbFilesStart instance to start the database initialization for dbfiles. Must be called when the database initialization is complete.

Parameters

class UTask * user UTask user for the callbacks for task completion.
class IDatabase * database The IDatabase instance of the database connection.
const char * folder A foreign key definition if the App handles folder management itself (e.g. "BIGINT REFERENCES messages(id) ON DELETE CASCADE"). A value of null indicates that dbfiles should do folder management.
It is important to use the ON DELETE CASCADE functionality of the foreign key, otherwise the files/folders can't be deleted correctly!

Return value

An instance of class IDbFilesStart.

Remarks

If dbfiles does the folder management, the root folder alway uses the id 1 (DBFILES_ROOT).
Put
Creates an IDbFilesPut instance to put a new file into the database.

Parameters

class UTask * user UTask user for the callbacks for task completion.
const char * name Name of the file. If dbfiles does folder management the name, folder combination must be unique, otherwise there are no constraints for the file name.
ulong64 folder Reference in which folder this file is to be put.
bool is_folder Only for folder management. If true a folder is put, not a file.
bool append If true, data is appended to a file if the file already exists. Note that the file size is -1 while a file is opened for writing/appending so a concurrent read on this file won't be possible.

Return value

An instance of class IDbFilesPut.
Get
Creates an IDbFilesGet instance to get a file from the database.

Parameters

class UTask * user UTask user for the callbacks for task completion.
ulong64 id id of the file to get (can be retrieved with IDbFilesList or IDbFilesPathInfo).
unsigned int offset Offset inside the file at which to start reading.
bool progress The flag progress is used to indicate that TaskProgress shall be called in any case when the data is available, even if this is the last block of data. In this case TaskComplete is called after the next Read() call, when the file is closed..

Return value

An instance of class IDbFilesGet.
Del
Creates an IDbFilesDel instance to get a file from the database.

Parameters

class UTask * user UTask user for the callbacks for task completion.
ulong64 id id of the file to delete (can be retrieved with IDbFilesList or IDbFilesPathInfo).

Return value

An instance of class IDbFilesDel.
Move
Creates an IDbFilesMove instance to move a file or folder.

Parameters

class UTask * user UTask user for the callbacks for task completion.
ulong64 id id of the file or folder to move (can be retrieved with IDbFilesList or IDbFilesPathInfo).
const char * name The new name of the file or folder.
ulong64 newFolder id of the new folder to move to (can be retrieved with IDbFilesList or IDbFilesPathInfo).

Return value

An instance of class IDbFilesMove.
List
Creates an IDbFilesList instance to list files and folders inside a folder.

Parameters

class UTask * user UTask user for the callbacks for task completion.
ulong64 folder id of the folder to list (can be retrieved IDbFilesPathInfo).
ulong64 limit Number of entries to get. If a limit is set (limit!=0), all the files of the folder are read into memory.
ulong64 idOffset Offset for the id of the file. Only files with id greater than idOffset are delivered. The file list is ordered by file id.

Return value

An instance of class IDbFilesList.
PathInfo
Creates an IDbFilesPathInfo instance to get information about a file or folder.

Parameters

class UTask * user UTask user for the callbacks for task completion.
const char * path File path to be tested. Should not start with '/'.

Return value

An instance of class IDbFilesPathInfo.

IDbFilesStart

Starts the database initialization for dbfiles. Must be called when the database initialization is complete. The ITask Start() method is implicitly called!
class IDbFilesStart {
public:
};

IDbFilesGet

Gets a file from the database. The ITask Start() method is implicitly called!
After TaskProgress (or TaskFinished) Get can be called to get the read data.
With Read the next data can be requested, which will be indicated with TaskProgress again. If no more data is available TaskComplete is called.
TaskFailed is called on errors.

class IDbFilesGet {
public:
    void Get(const byte * & data, unsigned & length);
    void Read(bool last=false);
    void Stop();
};

Public functions

Get
Get the read data. The data pointer is set to a pointer allocated by dbfiles and length is set to the length of the data. Do not free data, as data is handled by IDbFilesGet!

Parameters

const byte * & data Will be filled with available data.
unsigned & length Will be set to the length of the available data.
Read
Initiate next read.

Parameters

bool last last is used to indicate that this is the last data to be read even if there is more data available.
Stop
Stops the current operations which will trigger an asynchronous TaskComplete/TaskFailed.

IDbFilesPut

Puts a new file into the database. The ITask Start() method is implicitly called!
After the TaskComplete or TaskProgress callbacks, the function call Get can be used to read the id of the new file.
After TaskProgress the function Write can be called to write data to the file. This creates a new row in the files_data table. TaskFailed is called on errors.

class IDbFilesPut {
public:
    ulong64 Get();
    void Write(const byte * data, unsigned length, bool last);
    const char * GetName();
    void Stop();
};

Public functions

Get

Return value

The id of the file.

Remarks

Can be first called after TaskProgress/TaskComplete.
Write
Can be called after TaskProgress to write data to the file. This creates a new row in the files_data table.

Parameters

const byte * data The data to write.
unsigned length The length of the data to write.
bool last last is used to indicate that this is the last data to write, so TaskComplete is called afterwards.
Stop
Stops the current operation which will trigger an asynchronous TaskComplete/TaskFailed.

IDbFilesDel

Deletes a file from the database. The ITask Start() method is implicitly called!
TaskFailed is called on errors.

class IDbFilesDel {
public:
};

IDbFilesMove

Moves a file inside the database. The ITask Start() method is implicitly called!
TaskFailed is called on errors.

class IDbFilesMove {
public:
};

IDbFilesList

Gets the list of files assigned to a given folder. The ITask Start() method is implicitly called!

If a limit is set, TaskComplete indicates that the list is availabe. The entries in the list can be read with multiple calls to Get until true is returned.

If no limit is set the entries are read one by one. TaskProgress indicates that an entry is available, which can be read with Get.
With Next the next entry can be requested. If there are no more entries available TaskComplete is called.

class IDbFilesList {
public:
    bool Get(ulong64 & id, const char * & name, unsigned & length, bool & is_folder, ulong64 * created = 0, ulong64 * modified = 0);
    void Next();
    void Stop();
};

Public functions

Get

Parameters

ulong64 & id The id of the file or folder.
const char * & name
unsigned & length The file size.
bool & is_folder true if it's a folder.
ulong64 * created Creation time.
ulong64 * modified Last modification time.

Return value

true indicates that this was the last file in the list.
Next
Starts reading the next item in the list.
Stop
Stops the current operation which will trigger an asynchronous TaskComplete/TaskFailed.

IDbFilesPathInfo

Gets information of a file or folder.
class IDbFilesList {
public:
    void Get(ulong64 & id, const char * & name, unsigned & length, bool & is_folder, ulong64 * created = 0, ulong64 * modified = 0);
    void GetFailed(ulong64 & id, bool & is_folder, const char * & path);
};

Public functions

Get
Function which can be called from TaskComplete, to get the result.

Parameters

ulong64 & id The id of the file or folder.
const char * & name
unsigned & length The file size.
bool & is_folder true if it's a folder.
ulong64 * created Creation time.
ulong64 * modified Last modification time.
GetFailed
Function which can be called from TaskFailed, to get the result.

Parameters

ulong64 & id The id of the last part of the path, which was found.
bool & is_folder true if it's a folder.
const char * & path The path as it was found.

Data types

DBFILES_ROOT

#define DBFILES_ROOT 1

Overview

DBFILES_ROOT is used to define the root folder id.

DBFILES_CRYPT_KEY_LEN

#define DBFILES_CRYPT_KEY_LEN 32

Overview

DBFILES_CRYPT_KEY_LEN is used to define the length of the cryptKey, if files are stored encrypted.