flat_file_dictionary_handler.c File Reference

#include "flat_file_dictionary_handler.h"

Description

Function definitions at the dictionary interface level for the flat file store.

Author

Eric Huang

Copyright

Copyright 2017 The University of British Columbia, IonDB Project Contributors (see AUTHORS.md)

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions are met:

1.Redistributions of source code must retain the above copyright notice,

this list of conditions and the following disclaimer.

2.Redistributions in binary form must reproduce the above copyright notice,

this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

3.Neither the name of the copyright holder nor the names of its contributors

may be used to endorse or promote products derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"

AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Definition in file flat_file_dictionary_handler.c.

Include dependency graph for flat_file_dictionary_handler.c:

Functions
ion_cursor_status_t	ffdict_next (ion_dict_cursor_t *cursor, ion_record_t *record)
	Fetches the next record to be returned from a cursor that has already been initialized. More...
void	ffdict_destroy_cursor (ion_dict_cursor_t **cursor)
	Destroys and frees the given cursor. More...
ion_err_t	ffdict_open_dictionary (ion_dictionary_handler_t *handler, ion_dictionary_t *dictionary, ion_dictionary_config_info_t *config, ion_dictionary_compare_t compare)
	Re-instances a previously created flat file store instance and prepares it to be used again. More...
ion_err_t	ffdict_close_dictionary (ion_dictionary_t *dictionary)
	Closes this flat file store and persists everything to disk to be brought back later using dictionary_open. More...
ion_err_t	ffdict_find (ion_dictionary_t *dictionary, ion_predicate_t *predicate, ion_dict_cursor_t **cursor)
	Initializes a cursor query and returns an allocated cursor object. More...
void	ffdict_init (ion_dictionary_handler_t *handler)
	Given the handler instance, bind the appropriate flat file functions. More...
ion_status_t	ffdict_insert (ion_dictionary_t *dictionary, ion_key_t key, ion_value_t value)
	Given a record ( key, value ), insert it into the dictionary. More...
ion_status_t	ffdict_get (ion_dictionary_t *dictionary, ion_key_t key, ion_value_t value)
	Performs a "get" operation on the dictionary to retrieve a single record. More...
ion_err_t	ffdict_create_dictionary (ion_dictionary_id_t id, ion_key_type_t key_type, ion_key_size_t key_size, ion_value_size_t value_size, ion_dictionary_size_t dictionary_size, ion_dictionary_compare_t compare, ion_dictionary_handler_t *handler, ion_dictionary_t *dictionary)
	Creates an instance of a flat file backed dictionary. More...
ion_status_t	ffdict_delete (ion_dictionary_t *dictionary, ion_key_t key)
	Removes all instances of any record with key equal to key. More...
ion_err_t	ffdict_delete_dictionary (ion_dictionary_t *dictionary)
	Cleans up all files created by the dictionary, and frees any allocated memory. More...
ion_err_t	ffdict_destroy_dictionary (ion_dictionary_id_t id)
	Cleans up all files created by the dictionary, and frees any allocated memory, for an already closed dictionary. More...
ion_status_t	ffdict_update (ion_dictionary_t *dictionary, ion_key_t key, ion_value_t value)
	Updates all records stored at key to have value equal to value. More...

Function Documentation

ion_err_t ffdict_close_dictionary

(

ion_dictionary_t *

dictionary

)

Closes this flat file store and persists everything to disk to be brought back later using dictionary_open.

Parameters

[in]

dictionary

Which instance of a flat file store to close.

Returns

The resuling status of the operation.

Definition at line 178 of file flat_file_dictionary_handler.c.

  {
    ion_err_t err = flat_file_close((ion_flat_file_t *) dictionary->instance);

    free(dictionary->instance);
    dictionary->instance = NULL;

    if (err_ok != err) {
        return err;
    }

    return err_ok;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t ffdict_create_dictionary	(	ion_dictionary_id_t	id,
		ion_key_type_t	key_type,
		ion_key_size_t	key_size,
		ion_value_size_t	value_size,
		ion_dictionary_size_t	dictionary_size,
		ion_dictionary_compare_t	compare,
		ion_dictionary_handler_t *	handler,
		ion_dictionary_t *	dictionary
	)

Creates an instance of a flat file backed dictionary.

Parameters

[in]	id	The ID to assign to the dictionary. This is either user defined or is given by the master table.
[in]	key_type	The category of key used by the dictionary. See ION_KEY_TYPE for more information.
[in]	key_size	The size of the keys used for this dictionary, specified in bytes. It is strongly recommended to use a sizeof() directive to specify this size to avoid painful problems.
[in]	value_size	Same as above, for values.
[in]	dictionary_size	Designates how many records we buffer in memory. Higher means better overall performance at the cost of increased memory usage.
[in]	compare	The function pointer that designates how to compare two keys. This is given by the upper dictionary layers.
[in]	handler	The allocated, initialized handler that will be bound to the dictionary instance.
[in]	dictionary	The allocated dictionary instance that we are going to initialize.

Returns

The resulting status of the operation.

Definition at line 398 of file flat_file_dictionary_handler.c.

  {
    dictionary->instance = malloc(sizeof(ion_flat_file_t));

    if (NULL == dictionary->instance) {
        return err_out_of_memory;
    }

    dictionary->instance->compare   = compare;
    dictionary->instance->type      = dictionary_type_flat_file_t;

    ion_err_t result = flat_file_initialize((ion_flat_file_t *) dictionary->instance, id, key_type, key_size, value_size, dictionary_size);

    if ((err_ok == result) && (NULL != handler)) {
        dictionary->handler = handler;
    }

    return result;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t ffdict_delete	(	ion_dictionary_t *	dictionary,
		ion_key_t	key
	)

Removes all instances of any record with key equal to key.

Parameters

[in]	dictionary	Which dictionary to delete from.
[in]	key	Designated key to look for and remove.

Returns

The resulting status of the operation.

Definition at line 427 of file flat_file_dictionary_handler.c.

  {
    return flat_file_delete((ion_flat_file_t *) dictionary->instance, key);
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t ffdict_delete_dictionary

(

ion_dictionary_t *

dictionary

)

Cleans up all files created by the dictionary, and frees any allocated memory.

Parameters

[in]

dictionary

Which dictionary instance to delete.

Returns

The resulting status of the operation.

Definition at line 435 of file flat_file_dictionary_handler.c.

  {
    ion_err_t result = flat_file_destroy((ion_flat_file_t *) dictionary->instance);

    free(dictionary->instance);
    dictionary->instance = NULL;
    return result;
}

Here is the call graph for this function:

Here is the caller graph for this function:

void ffdict_destroy_cursor

(

ion_dict_cursor_t **

cursor

)

Destroys and frees the given cursor.

This function should not be called directly, but instead accessed through the interface the cursor object.

Parameters

[in]

cursor

Which cursor to destroy.

Definition at line 134 of file flat_file_dictionary_handler.c.

  {
    (*cursor)->predicate->destroy(&(*cursor)->predicate);
    free(*cursor);
    *cursor = NULL;
}

Here is the caller graph for this function:

ion_err_t ffdict_destroy_dictionary

(

ion_dictionary_id_t

id

)

Cleans up all files created by the dictionary, and frees any allocated memory, for an already closed dictionary.

Parameters

[in]

id

The identifier identifying the dictionary to delete.

Returns

The resulting status of the operation.

Definition at line 446 of file flat_file_dictionary_handler.c.

  {
    char filename[ION_MAX_FILENAME_LENGTH];

    dictionary_get_filename(id, "ffs", filename);

    if (0 != fremove(filename)) {
        return err_file_delete_error;
    }

    return err_ok;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t ffdict_find	(	ion_dictionary_t *	dictionary,
		ion_predicate_t *	predicate,
		ion_dict_cursor_t **	cursor
	)

Initializes a cursor query and returns an allocated cursor object.

Given a predicate that was previously initialized by dictionary_build_predicate, Build a cursor object that acts as the iterator for the desired query through the dictionary.

Parameters

[in]	dictionary	Which dictionary to query on.
[in]	predicate	An allocated, initialized predicate object that defines the parameters of the query.
[out]	cursor	A cursor pointer should be initialized to NULL, and then given to this function. This function shall allocate appropriate memory and redirect cursor to point to the allocated memory. NOTE: Anything originally pointed to by this cursor will effectively be lost, if there is no other reference to said thing.

Returns

The resulting status of the operation.

Definition at line 209 of file flat_file_dictionary_handler.c.

  {
    *cursor = malloc(sizeof(ion_flat_file_cursor_t));

    ion_flat_file_t *flat_file = (ion_flat_file_t *) dictionary->instance;

    if (NULL == *cursor) {
        return err_out_of_memory;
    }

    (*cursor)->dictionary   = dictionary;
    (*cursor)->status       = cs_cursor_uninitialized;

    (*cursor)->destroy      = ffdict_destroy_cursor;
    (*cursor)->next         = ffdict_next;

    (*cursor)->predicate    = malloc(sizeof(ion_predicate_t));

    if (NULL == (*cursor)->predicate) {
        free(*cursor);
        return err_out_of_memory;
    }

    (*cursor)->predicate->type      = predicate->type;
    (*cursor)->predicate->destroy   = predicate->destroy;

    ion_key_size_t key_size = dictionary->instance->record.key_size;

    switch (predicate->type) {
        case predicate_equality: {
            ion_key_t target_key = predicate->statement.equality.equality_value;

            (*cursor)->predicate->statement.equality.equality_value = malloc(key_size);

            if (NULL == (*cursor)->predicate->statement.equality.equality_value) {
                free((*cursor)->predicate);
                free(*cursor);
                return err_out_of_memory;
            }

            memcpy((*cursor)->predicate->statement.equality.equality_value, target_key, key_size);

            ion_fpos_t          loc         = -1;
            ion_flat_file_row_t row;
            ion_err_t           scan_result = flat_file_scan(flat_file, -1, &loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_key_match, target_key);

            if (err_file_hit_eof == scan_result) {
                /* If this happens, that means the target key doesn't exist */
                (*cursor)->status = cs_end_of_results;
                return err_ok;
            }
            else if (err_ok == scan_result) {
                (*cursor)->status = cs_cursor_initialized;

                ion_flat_file_cursor_t *flat_file_cursor = (ion_flat_file_cursor_t *) (*cursor);

                flat_file_cursor->current_location = loc;
                return err_ok;
            }
            else {
                /* File scan hit an error condition */
                return scan_result;
            }

            break;
        }

        case predicate_range: {
            (*cursor)->predicate->statement.range.lower_bound = malloc(key_size);

            if (NULL == (*cursor)->predicate->statement.range.lower_bound) {
                free((*cursor)->predicate);
                free(*cursor);
                return err_out_of_memory;
            }

            memcpy((*cursor)->predicate->statement.range.lower_bound, predicate->statement.range.lower_bound, key_size);

            (*cursor)->predicate->statement.range.upper_bound = malloc(key_size);

            if (NULL == (*cursor)->predicate->statement.range.upper_bound) {
                free((*cursor)->predicate->statement.range.lower_bound);
                free((*cursor)->predicate);
                free(*cursor);
                return err_out_of_memory;
            }

            memcpy((*cursor)->predicate->statement.range.upper_bound, predicate->statement.range.upper_bound, key_size);

            /* Find the first satisfactory key. */
            ion_fpos_t          loc         = -1;
            ion_flat_file_row_t row;
            ion_err_t           scan_result = flat_file_scan(flat_file, -1, &loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_within_bounds, (*cursor)->predicate->statement.range.lower_bound, (*cursor)->predicate->statement.range.upper_bound);

            if (err_file_hit_eof == scan_result) {
                /* This means the returned node is smaller than the lower bound, which means that there are no valid records to return */
                (*cursor)->status = cs_end_of_results;
                return err_ok;
            }
            else if (err_ok == scan_result) {
                (*cursor)->status = cs_cursor_initialized;

                ion_flat_file_cursor_t *flat_file_cursor = (ion_flat_file_cursor_t *) (*cursor);

                flat_file_cursor->current_location = loc;
                return err_ok;
            }
            else {
                /* Scan failed due to external error */
                return scan_result;
            }

            break;
        }

        case predicate_all_records: {
            ion_flat_file_cursor_t *flat_file_cursor    = (ion_flat_file_cursor_t *) (*cursor);

            ion_fpos_t          loc                     = -1;
            ion_flat_file_row_t row;
            ion_err_t           scan_result             = flat_file_scan(flat_file, -1, &loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_not_empty);

            if (err_file_hit_eof == scan_result) {
                (*cursor)->status = cs_end_of_results;
            }
            else if (err_ok == scan_result) {
                flat_file_cursor->current_location  = loc;
                (*cursor)->status                   = cs_cursor_initialized;
            }
            else {
                /* Scan failure */
                return scan_result;
            }

            return err_ok;
            break;
        }

        case predicate_predicate: {
            break;
        }

        default: {
            return err_invalid_predicate;
            break;
        }
    }

    return err_ok;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t ffdict_get	(	ion_dictionary_t *	dictionary,
		ion_key_t	key,
		ion_value_t	value
	)

Performs a "get" operation on the dictionary to retrieve a single record.

Given a key, returns the associated value stored under that key. If there are duplicate records all with the same key, the exact one that is returned is undefined. If you need to get all records stored under a specific key, use a cursor query.

Parameters

[in]	dictionary	Which dictionary to perform the operation on.
[in]	key	The desired search key.
[out]	value	The output location in which to write the returned value. This space must be allocated by the user to at least value_size bytes, as originally defined on dictionary creation.

Returns

The resulting status of the operation.

Definition at line 389 of file flat_file_dictionary_handler.c.

  {
    return flat_file_get((ion_flat_file_t *) dictionary->instance, key, value);
}

Here is the call graph for this function:

Here is the caller graph for this function:

void ffdict_init

(

ion_dictionary_handler_t *

handler

)

Given the handler instance, bind the appropriate flat file functions.

Parameters

[in]

handler

The handler is assumed to be memory that is allocated and initialized by the user.

Definition at line 364 of file flat_file_dictionary_handler.c.

  {
    handler->insert             = ffdict_insert;
    handler->create_dictionary  = ffdict_create_dictionary;
    handler->get                = ffdict_get;
    handler->update             = ffdict_update;
    handler->find               = ffdict_find;
    handler->remove             = ffdict_delete;
    handler->delete_dictionary  = ffdict_delete_dictionary;
    handler->destroy_dictionary = ffdict_destroy_dictionary;
    handler->open_dictionary    = ffdict_open_dictionary;
    handler->close_dictionary   = ffdict_close_dictionary;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t ffdict_insert	(	ion_dictionary_t *	dictionary,
		ion_key_t	key,
		ion_value_t	value
	)

Given a record ( key, value ), insert it into the dictionary.

Parameters

[in]	dictionary	The initialized dictionary instance we want to insert into.
[in]	key	The key portion of the record to be inserted.
[in]	value	The value portion of the record to be inserted.

Returns

The resulting status of the operation.

Definition at line 380 of file flat_file_dictionary_handler.c.

  {
    return flat_file_insert((ion_flat_file_t *) dictionary->instance, key, value);
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_cursor_status_t ffdict_next	(	ion_dict_cursor_t *	cursor,
		ion_record_t *	record
	)

Fetches the next record to be returned from a cursor that has already been initialized.

The returned record is written back to record, and then the cursor is advanced to the next record. The returned status code signifies whether or not there are more results to traverse. This function should not be called directly, but instead will be bound to the cursor like a method.

Parameters

[in]	cursor	Which cursor to fetch results from.
[out]	record	An initialized record struct with the key and value appropriately allocated to fit the returned key and value. This function will write back data to the struct.

Returns

The resulting status of the operation.

Definition at line 53 of file flat_file_dictionary_handler.c.

  {
    ion_flat_file_t         *flat_file          = (ion_flat_file_t *) cursor->dictionary->instance;
    ion_flat_file_cursor_t  *flat_file_cursor   = (ion_flat_file_cursor_t *) cursor;

    if (cursor->status == cs_cursor_uninitialized) {
        return cursor->status;
    }
    else if (cursor->status == cs_end_of_results) {
        return cursor->status;
    }
    else if ((cursor->status == cs_cursor_initialized) || (cursor->status == cs_cursor_active)) {
        if (cursor->status == cs_cursor_active) {
            ion_flat_file_row_t throwaway_row;
            ion_err_t           err = err_uninitialized;

            switch (cursor->predicate->type) {
                case predicate_equality: {
                    err = flat_file_scan(flat_file, flat_file_cursor->current_location + 1, &flat_file_cursor->current_location, &throwaway_row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_key_match, cursor->predicate->statement.equality.equality_value);

                    break;
                }

                case predicate_range: {
                    err = flat_file_scan(flat_file, flat_file_cursor->current_location + 1, &flat_file_cursor->current_location, &throwaway_row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_within_bounds, cursor->predicate->statement.range.lower_bound, cursor->predicate->statement.range.upper_bound);

                    break;
                }

                case predicate_all_records: {
                    err = flat_file_scan(flat_file, flat_file_cursor->current_location + 1, &flat_file_cursor->current_location, &throwaway_row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_not_empty);

                    break;
                }

                case predicate_predicate: {
                    break;
                }
            }

            if (err_file_hit_eof == err) {
                cursor->status = cs_end_of_results;
                return cursor->status;
            }
            else if (err_ok != err) {
                cursor->status = cs_possible_data_inconsistency;
                return cursor->status;
            }
        }
        else {
            /* The status is cs_cursor_initialized */
            cursor->status = cs_cursor_active;
        }

        ion_flat_file_row_t row;
        ion_err_t           err = flat_file_read_row(flat_file, flat_file_cursor->current_location, &row);

        if (err_ok != err) {
            return cs_invalid_index;
        }

        /*Copy both key and value into user provided struct */
        memcpy(record->key, row.key, cursor->dictionary->instance->record.key_size);
        memcpy(record->value, row.value, cursor->dictionary->instance->record.value_size);

        return cursor->status;
    }

    return cs_invalid_cursor;
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t ffdict_open_dictionary	(	ion_dictionary_handler_t *	handler,
		ion_dictionary_t *	dictionary,
		ion_dictionary_config_info_t *	config,
		ion_dictionary_compare_t	compare
	)

Re-instances a previously created flat file store instance and prepares it to be used again.

Parameters

[in]	handler	A handler that must be bound with the flat file's functions.
[in]	dictionary	A dictionary that is allocated but not initialized. We will store the details of the re-instanced flat file store in here.
[in]	config	The configuration parameters previously used by the flat file store we are opening. This must either be provided directly if dictionaries are being managed manually, or will be provided by the dictionary manager that is currently in use.
[in]	compare	The comparison function that will be given by higher layers, based on the destined key type.

Returns

The resulting status of the operation.

Definition at line 161 of file flat_file_dictionary_handler.c.

  {
    return ffdict_create_dictionary(config->id, config->type, config->key_size, config->value_size, config->dictionary_size, compare, handler, dictionary);
}

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t ffdict_update	(	ion_dictionary_t *	dictionary,
		ion_key_t	key,
		ion_value_t	value
	)

Updates all records stored at key to have value equal to value.

If no records are stored at key, then an "upsert" (insert instead of update) is performed.

Parameters

[in]	dictionary	Which dictionary to update.
[in]	key	Target key to update.
[in]	value	New value to update to.

Returns

The resulting status of the operation.

Definition at line 461 of file flat_file_dictionary_handler.c.

  {
    return flat_file_update((ion_flat_file_t *) dictionary->instance, key, value);
}

Here is the call graph for this function:

Here is the caller graph for this function: