flat_file.h File Reference
#include "flat_file_types.h"

Description

Implementation specific declarations for the flat file store.

Author
Eric Huang
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1.Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2.Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3.Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Definition in file flat_file.h.

Include dependency graph for flat_file.h:
This graph shows which files directly or indirectly include this file:

Functions

ion_err_t flat_file_initialize (ion_flat_file_t *flat_file, ion_dictionary_id_t id, ion_key_type_t key_type, ion_key_size_t key_size, ion_value_size_t value_size, ion_dictionary_size_t dictionary_size)
 Initializes the flat file implementation and creates all necessary files. More...
 
ion_err_t flat_file_destroy (ion_flat_file_t *flat_file)
 Destroys and cleans up any implementation specific memory or files. More...
 
ion_status_t flat_file_insert (ion_flat_file_t *flat_file, ion_key_t key, ion_value_t value)
 Inserts the given record into the flat file store. More...
 
ion_status_t flat_file_get (ion_flat_file_t *flat_file, ion_key_t key, ion_value_t value)
 Fetches the record stored with the given key. More...
 
ion_status_t flat_file_delete (ion_flat_file_t *flat_file, ion_key_t key)
 Deletes all records stored with the given key. More...
 
ion_status_t flat_file_update (ion_flat_file_t *flat_file, ion_key_t key, ion_value_t value)
 Updates all records stored with the given key to have value. More...
 
ion_err_t flat_file_close (ion_flat_file_t *flat_file)
 Closes and frees any memory associated with the flat file. More...
 
ion_err_t flat_file_scan (ion_flat_file_t *flat_file, ion_fpos_t start_location, ion_fpos_t *location, ion_flat_file_row_t *row, ion_byte_t scan_direction, ion_flat_file_predicate_t predicate,...)
 Performs a linear scan of the flat file writing the first location seen that satisfies the given predicate to location. More...
 
ion_boolean_t flat_file_predicate_key_match (ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
 Predicate function to return any row that has an exact match to the given target key. More...
 
ion_boolean_t flat_file_predicate_within_bounds (ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
 Predicate function to return any row that has a key such that lower_bound <= key <= upper_bound holds true. More...
 
ion_boolean_t flat_file_predicate_not_empty (ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
 Predicate function to return any row that is not empty or deleted. More...
 
ion_err_t flat_file_read_row (ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
 Reads the row specified by the given location into the buffer. More...
 
ion_err_t flat_file_binary_search (ion_flat_file_t *flat_file, ion_key_t target_key, ion_fpos_t *location)
 Performs a binary search for the given target_key, returning to location the first-less-than-or-equal key within the flat file. This can only be used if sorted_mode is enabled within the flat file. More...
 

Function Documentation

ion_err_t flat_file_binary_search ( ion_flat_file_t flat_file,
ion_key_t  target_key,
ion_fpos_t location 
)

Performs a binary search for the given target_key, returning to location the first-less-than-or-equal key within the flat file. This can only be used if sorted_mode is enabled within the flat file.

In the case of duplicates, this function will scroll to the beginning of the block of duplicates, before writing back to location. As a result, it is guaranteed that the returned index points to the first key in a contiguous block of duplicate keys. If no key in the flat file satisfies the condition of being less-than-or-equal, then -1 is written back to location. This function will only return records that are not deleted.

Parameters
[in]flat_fileWhich flat file instance to search within.
[in]target_keyDesired key to search for.
[out]locationFound location to write back into. Must be allocated by caller.
Returns
Resulting status of the binary search operation.

Definition at line 638 of file flat_file.c.

642  {
643  if (!flat_file->sorted_mode) {
645  }
646 
647  ion_err_t err;
649  ion_fpos_t low_idx = 0;
650  ion_fpos_t high_idx = (flat_file->eof_position - flat_file->start_of_data) / flat_file->row_size - 1;
651  ion_fpos_t mid_idx;
652 
653  if (high_idx < 0) {
654  /* We're empty, short circuit */
655  *location = -1;
656  return err_item_not_found;
657  }
658 
659  while (low_idx < high_idx) {
660  mid_idx = low_idx + (high_idx - low_idx) / 2;
661  err = flat_file_read_row(flat_file, mid_idx, &row);
662 
663  if (err_ok != err) {
664  return err;
665  }
666 
667  char comp_result = flat_file->super.compare(target_key, row.key, flat_file->super.record.key_size);
668 
669  if (comp_result > 0) {
670  low_idx = mid_idx + 1;
671  }
672  else if (comp_result < 0) {
673  high_idx = mid_idx - 1;
674  }
675  else {
676  /* Match found, scroll to beginning of (potential) duplicate block and return */
677  ion_fpos_t last_dup_idx;
678  ion_fpos_t dup_idx = mid_idx;
679 
680  do {
681  last_dup_idx = dup_idx;
682  dup_idx--;
683  err = flat_file_read_row(flat_file, dup_idx, &row);
684 
685  if (err_ok != err) {
686  return err;
687  }
688  } while (dup_idx >= 0 && 0 == flat_file->super.compare(row.key, target_key, flat_file->super.record.key_size));
689 
690  *location = last_dup_idx;
691  return err_ok;
692  }
693  }
694 
695  /* If we reach here, then we fell through the loop - do check and adjust for LEQ as necessary */
696  err = flat_file_read_row(flat_file, low_idx, &row);
697 
698  if (err_ok != err) {
699  return err;
700  }
701 
702  if (flat_file->super.compare(row.key, target_key, flat_file->super.record.key_size) > 0) {
703  low_idx--;
704  }
705 
706  *location = low_idx;
707  return low_idx >= 0 ? err_ok : err_item_not_found;
708 }
ion_record_info_t record
Container for the rows written in the flat file data file.
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
ion_err_t flat_file_read_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Reads the row specified by the given location into the buffer.
Definition: flat_file.c:350
ion_fpos_t start_of_data
ion_fpos_t eof_position
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
ion_boolean_t sorted_mode
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_dictionary_compare_t compare

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t flat_file_close ( ion_flat_file_t flat_file)

Closes and frees any memory associated with the flat file.

Parameters
flat_fileWhich flat file to close.
Returns
Status of closure.

Definition at line 624 of file flat_file.c.

626  {
627  free(flat_file->buffer);
628  flat_file->buffer = NULL;
629 
630  if (0 != fclose(flat_file->data_file)) {
631  return err_file_close_error;
632  }
633 
634  return err_ok;
635 }
ion_byte_t * buffer

Here is the caller graph for this function:

ion_status_t flat_file_delete ( ion_flat_file_t flat_file,
ion_key_t  key 
)

Deletes all records stored with the given key.

Parameters
[in]flat_fileWhich flat file to delete in.
[in]keySpecified key to find and delete.
Returns
Resulting status of the operation.
See also
ffdict_delete

Definition at line 495 of file flat_file.c.

498  {
499  if (flat_file->sorted_mode) {
501  }
502 
505  ion_err_t err;
506  ion_fpos_t loc = -1;
507 
508  while (err_ok == (err = flat_file_scan(flat_file, loc, &loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_key_match, key))) {
509  ion_fpos_t last_record_offset = flat_file->eof_position - flat_file->row_size;
510  ion_flat_file_row_t last_row;
511  ion_fpos_t last_record_index = (last_record_offset - flat_file->start_of_data) / flat_file->row_size;
512  ion_err_t row_err;
513 
514  /* If the last index and the loc are the same, then we can just move the eof position. Saves a read/write. */
515  if (last_record_index != loc) {
516  row_err = flat_file_read_row(flat_file, last_record_index, &last_row);
517 
518  if (err_ok != row_err) {
519  status.error = row_err;
520  return status;
521  }
522 
523  row_err = flat_file_write_row(flat_file, loc, &last_row);
524 
525  if (err_ok != row_err) {
526  status.error = row_err;
527  return status;
528  }
529  }
530 
531  /* Set last row to be empty just for sanity reasons. */
532  row_err = flat_file_write_row(flat_file, last_record_index, &(ion_flat_file_row_t) { ION_FLAT_FILE_STATUS_EMPTY, NULL, NULL });
533 
534  if (err_ok != row_err) {
535  status.error = row_err;
536  return status;
537  }
538 
539  /* Soft truncate the file by bumping the eof position back to cut off the last record. */
540  flat_file->eof_position = last_record_offset;
541  status.count++;
542 
543  /* No location movement is done here, since we need to check the row we just swapped in to see if it is
544  also a match. */
545  }
546 
547  status.error = err_ok;
548 
549  if (((err == err_file_hit_eof) || (-1 == loc)) && (status.count == 0)) {
550  status.error = err_item_not_found;
551  }
552  else if (err != err_file_hit_eof) {
553  status.error = err;
554  }
555 
556  return status;
557 }
ion_err_t flat_file_write_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Writes the given row out to the data file.
Definition: flat_file.c:321
#define ION_STATUS_ERROR(error)
Definition: kv_system.h:110
Container for the rows written in the flat file data file.
#define ION_FLAT_FILE_STATUS_EMPTY
Signifies that this row in the flat file is currently empty and is okay to be overwritten.
#define key(k)
Definition: bpp_tree.c:75
ion_err_t error
Definition: kv_system.h:291
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
ion_err_t flat_file_read_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Reads the row specified by the given location into the buffer.
Definition: flat_file.c:350
ion_fpos_t start_of_data
ion_fpos_t eof_position
ion_err_t flat_file_scan(ion_flat_file_t *flat_file, ion_fpos_t start_location, ion_fpos_t *location, ion_flat_file_row_t *row, ion_byte_t scan_direction, ion_flat_file_predicate_t predicate,...)
Performs a linear scan of the flat file writing the first location seen that satisfies the given pred...
Definition: flat_file.c:157
ion_result_count_t count
Definition: kv_system.h:293
#define ION_FLAT_FILE_SCAN_FORWARDS
Signals to flat_file_scan to scan in a forward direction.
ion_boolean_t sorted_mode
ion_boolean_t flat_file_predicate_key_match(ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
Predicate function to return any row that has an exact match to the given target key.
Definition: flat_file.c:280
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
#define ION_STATUS_INITIALIZE
Definition: kv_system.h:107
A status object that describes the result of a dictionary operation.
Definition: kv_system.h:290

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t flat_file_destroy ( ion_flat_file_t flat_file)

Destroys and cleans up any implementation specific memory or files.

Parameters
[in]flat_fileGiven flat file instance to destroy.
Returns
The resulting status of destruction.
See also
ffdict_delete_dictionary

Definition at line 134 of file flat_file.c.

136  {
137  ion_err_t err = flat_file_close(flat_file);
138 
139  if (err_ok != err) {
140  return err;
141  }
142 
143  char filename[ION_MAX_FILENAME_LENGTH];
144 
145  dictionary_get_filename(flat_file->super.id, "ffs", filename);
146 
147  if (0 != fremove(filename)) {
148  return err_file_delete_error;
149  }
150 
151  flat_file->data_file = NULL;
152 
153  return err_ok;
154 }
#define fremove(x)
Definition: kv_system.h:56
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
int dictionary_get_filename(ion_dictionary_id_t id, char *ext, char *filename)
Given the ID, implementation specific extension, and a buffer to write to, writes back the formatted ...
Definition: dictionary.c:41
ion_dictionary_id_t id
#define ION_MAX_FILENAME_LENGTH
Since the arduino conforms to 8.3 syntax, that&#39;s 8 + 3 = 11 + 1 (null terminator) characters...
Definition: kv_system.h:73
ion_dictionary_parent_t super
ion_err_t flat_file_close(ion_flat_file_t *flat_file)
Closes and frees any memory associated with the flat file.
Definition: flat_file.c:624

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t flat_file_get ( ion_flat_file_t flat_file,
ion_key_t  key,
ion_value_t  value 
)

Fetches the record stored with the given key.

Parameters
[in]flat_fileWhich flat file to look in.
[in]keySpecified key to look for.
[out]valueValue portion of the record to insert.
Returns
Resulting status of the operation.
See also
ffdict_get

Definition at line 439 of file flat_file.c.

443  {
445  ion_err_t err;
446  ion_fpos_t found_loc = -1;
448 
449  if (!flat_file->sorted_mode) {
450  err = flat_file_scan(flat_file, -1, &found_loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_key_match, key);
451 
452  if (err_ok != err) {
453  if (err_file_hit_eof == err) {
454  /* Alias the error since in this case, since hitting the EOF signifies */
455  /* that we didn't find what we were looking for */
456  status.error = err_item_not_found;
457  }
458  else {
459  /* This error takes priority since it is likely a file I/O issue */
460  status.error = err;
461  }
462 
463  return status;
464  }
465  }
466  else {
467  err = flat_file_binary_search(flat_file, key, &found_loc);
468 
469  if (err_ok != err) {
470  status.error = err;
471  return status;
472  }
473 
474  err = flat_file_read_row(flat_file, found_loc, &row);
475 
476  if (err_ok != err) {
477  status.error = err;
478  return status;
479  }
480 
481  if (0 != flat_file->super.compare(row.key, key, flat_file->super.record.key_size)) {
482  status.error = err_item_not_found;
483  return status;
484  }
485  }
486 
487  memcpy(value, row.value, flat_file->super.record.value_size);
488  status.error = err_ok;
489  status.count = 1;
490 
491  return status;
492 }
ion_err_t flat_file_binary_search(ion_flat_file_t *flat_file, ion_key_t target_key, ion_fpos_t *location)
Performs a binary search for the given target_key, returning to location the first-less-than-or-equal...
Definition: flat_file.c:638
ion_record_info_t record
Container for the rows written in the flat file data file.
#define key(k)
Definition: bpp_tree.c:75
ion_err_t error
Definition: kv_system.h:291
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
ion_err_t flat_file_read_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Reads the row specified by the given location into the buffer.
Definition: flat_file.c:350
ion_err_t flat_file_scan(ion_flat_file_t *flat_file, ion_fpos_t start_location, ion_fpos_t *location, ion_flat_file_row_t *row, ion_byte_t scan_direction, ion_flat_file_predicate_t predicate,...)
Performs a linear scan of the flat file writing the first location seen that satisfies the given pred...
Definition: flat_file.c:157
ion_result_count_t count
Definition: kv_system.h:293
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
#define ION_FLAT_FILE_SCAN_FORWARDS
Signals to flat_file_scan to scan in a forward direction.
ion_boolean_t sorted_mode
ion_boolean_t flat_file_predicate_key_match(ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
Predicate function to return any row that has an exact match to the given target key.
Definition: flat_file.c:280
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_dictionary_compare_t compare
ion_value_size_t value_size
Definition: kv_system.h:309
#define ION_STATUS_INITIALIZE
Definition: kv_system.h:107
A status object that describes the result of a dictionary operation.
Definition: kv_system.h:290

Here is the call graph for this function:

Here is the caller graph for this function:

ion_err_t flat_file_initialize ( ion_flat_file_t flat_file,
ion_dictionary_id_t  id,
ion_key_type_t  key_type,
ion_key_size_t  key_size,
ion_value_size_t  value_size,
ion_dictionary_size_t  dictionary_size 
)

Initializes the flat file implementation and creates all necessary files.

A check is done to see if this is actually an attempt to open a previously existing flat file instance. This (should) only happen when this is called from an open context instead of an initialize. The flat file supports a special mode called "sorted mode". This is an append only mode that assumes all keys come in monotonic non-decreasing order. In this mode, search operations are significantly faster, but the store does not support deletions while in sorted mode.

Parameters
[in]flat_fileGiven instance of a flat file struct to initialize. This must be allocated heap memory, as destruction will assume that it needs to be freed.
[in]idThe assigned ID of this dictionary instance.
[in]key_typeKey category to use for this instance.
[in]key_sizeKey size, in bytes used for this instance.
[in]value_sizeValue size, in bytes used for this instance.
[in]dictionary_sizeDictionary size is interpreted as how many records (key value pairs) are buffered. This should be given as somewhere between 1 (minimum) and the page size of the device you are working on.
Returns
The status of initialization.
See also
ffdict_create_dictionary

Definition at line 40 of file flat_file.c.

47  {
48  if (dictionary_size <= 0) {
49  /* Clamp the dictionary size since we always need at least 1 row to buffer */
50  dictionary_size = 1;
51  }
52 
53  flat_file->super.key_type = key_type;
54  flat_file->super.record.key_size = key_size;
55  flat_file->super.record.value_size = value_size;
56 
57  char filename[ION_MAX_FILENAME_LENGTH];
58  int actual_filename_length = dictionary_get_filename(id, "ffs", filename);
59 
60  if (actual_filename_length >= ION_MAX_FILENAME_LENGTH) {
61  return err_uninitialized;
62  }
63 
64  flat_file->sorted_mode = boolean_false;/* By default, we don't use sorted mode */
65  flat_file->num_buffered = dictionary_size;
66  flat_file->current_loaded_region = -1; /* No loaded region yet */
67 
68  flat_file->data_file = fopen(filename, "r+b");
69 
70  if (NULL == flat_file->data_file) {
71  /* The file did not exist - lets open to write */
72  flat_file->data_file = fopen(filename, "w+b");
73 
74  if (NULL == flat_file->data_file) {
75  /* Failed to open, even to create */
76  return err_file_open_error;
77  }
78  }
79 
80  /* For now, we don't have any header information. But we write some garbage there just so that
81  we can verify that the code to handle the header is working.*/
82  fwrite(&(int) { 0xADDE }, sizeof(int), 1, flat_file->data_file);
83  flat_file->start_of_data = ftell(flat_file->data_file);
84 
85  if (-1 == flat_file->start_of_data) {
86  fclose(flat_file->data_file);
87  return err_file_read_error;
88  }
89 
90  /* A record is laid out as: | STATUS | KEY | VALUE | */
91  /* Bytes: (1) (key_size) (value_size) */
92  flat_file->row_size = sizeof(ion_flat_file_row_status_t) + key_size + value_size;
93  flat_file->buffer = calloc(flat_file->num_buffered, flat_file->row_size);
94 
95  if (NULL == flat_file->buffer) {
96  fclose(flat_file->data_file);
97  return err_out_of_memory;
98  }
99 
100  if (0 != fseek(flat_file->data_file, 0, SEEK_END)) {
101  fclose(flat_file->data_file);
102  return err_file_bad_seek;
103  }
104 
105  flat_file->eof_position = ftell(flat_file->data_file);
106 
107  if (-1 == flat_file->eof_position) {
108  fclose(flat_file->data_file);
109  return err_file_read_error;
110  }
111 
112  /* Now move the eof to the last non-empty row in the file */
113  ion_fpos_t loc = -1;
116 
117  if ((err_ok != err) && (err_file_hit_eof != err)) {
118  fclose(flat_file->data_file);
119  return err;
120  }
121 
122  if (err_file_hit_eof == err) {
123  /* Then there are no occupied rows in the file. We'll set to the start of data. */
124  loc = -1;
125  }
126 
127  /* Move to its final position as one-past the position found. */
128  flat_file->eof_position = flat_file->start_of_data + (loc + 1) * flat_file->row_size;
129 
130  return err_ok;
131 }
ion_dictionary_size_t num_buffered
ion_record_info_t record
Container for the rows written in the flat file data file.
ion_fpos_t current_loaded_region
ion_byte_t ion_flat_file_row_status_t
This type describes the status flag within a flat file row.
#define ION_FLAT_FILE_SCAN_BACKWARDS
Signals to flat_file_scan to scan in a backward direction.
ion_byte_t * buffer
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
int dictionary_get_filename(ion_dictionary_id_t id, char *ext, char *filename)
Given the ID, implementation specific extension, and a buffer to write to, writes back the formatted ...
Definition: dictionary.c:41
ion_fpos_t start_of_data
ion_fpos_t eof_position
ion_err_t flat_file_scan(ion_flat_file_t *flat_file, ion_fpos_t start_location, ion_fpos_t *location, ion_flat_file_row_t *row, ion_byte_t scan_direction, ion_flat_file_predicate_t predicate,...)
Performs a linear scan of the flat file writing the first location seen that satisfies the given pred...
Definition: flat_file.c:157
#define ION_MAX_FILENAME_LENGTH
Since the arduino conforms to 8.3 syntax, that&#39;s 8 + 3 = 11 + 1 (null terminator) characters...
Definition: kv_system.h:73
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
ion_boolean_t sorted_mode
ion_key_type_t key_type
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_boolean_t flat_file_predicate_not_empty(ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
Predicate function to return any row that is not empty or deleted.
Definition: flat_file.c:268
ion_value_size_t value_size
Definition: kv_system.h:309

Here is the call graph for this function:

Here is the caller graph for this function:

ion_status_t flat_file_insert ( ion_flat_file_t flat_file,
ion_key_t  key,
ion_value_t  value 
)

Inserts the given record into the flat file store.

Parameters
[in]flat_fileWhich flat file to insert into.
[in]keyKey portion of the record to insert.
[in]valueValue portion of the record to insert.
Returns
Resulting status of insertion.
See also
ffdict_insert

Definition at line 388 of file flat_file.c.

392  {
394  ion_err_t err;
395  /* We can assume append-only insert here because our delete operation does a swap replacement, and
396  in sorted mode, we don't allow deletes - so there are no holes to fill. */
397  ion_fpos_t insert_loc = (flat_file->eof_position - flat_file->start_of_data) / flat_file->row_size;
398 
399  if (flat_file->sorted_mode) {
400  ion_fpos_t last_record_loc = (flat_file->eof_position - flat_file->start_of_data) / flat_file->row_size - 1;
402 
403  if (last_record_loc >= 0) {
404  err = flat_file_read_row(flat_file, last_record_loc, &row);
405 
406  if (err_ok != err) {
407  status.error = err;
408  return status;
409  }
410 
411  if (flat_file->super.compare(key, row.key, flat_file->super.record.key_size) < 0) {
413  return status;
414  }
415  }
416  }
417 
418  err = flat_file_write_row(flat_file, insert_loc, &(ion_flat_file_row_t) { ION_FLAT_FILE_STATUS_OCCUPIED, key, value });
419 
420  if (err_ok != err) {
421  status.error = err;
422  return status;
423  }
424 
425  /* Record new eof position */
426  flat_file->eof_position = ftell(flat_file->data_file);
427 
428  if (-1 == flat_file->eof_position) {
429  status.error = err_file_read_error;
430  return status;
431  }
432 
433  status.error = err_ok;
434  status.count = 1;
435  return status;
436 }
ion_err_t flat_file_write_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Writes the given row out to the data file.
Definition: flat_file.c:321
ion_record_info_t record
Container for the rows written in the flat file data file.
#define key(k)
Definition: bpp_tree.c:75
ion_err_t error
Definition: kv_system.h:291
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
ion_err_t flat_file_read_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Reads the row specified by the given location into the buffer.
Definition: flat_file.c:350
ion_fpos_t start_of_data
ion_fpos_t eof_position
ion_result_count_t count
Definition: kv_system.h:293
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
#define ION_FLAT_FILE_STATUS_OCCUPIED
Signifies that this row in the flat file is currently occupied and should not be overwritten.
ion_boolean_t sorted_mode
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_dictionary_compare_t compare
#define ION_STATUS_INITIALIZE
Definition: kv_system.h:107
A status object that describes the result of a dictionary operation.
Definition: kv_system.h:290

Here is the call graph for this function:

Here is the caller graph for this function:

ion_boolean_t flat_file_predicate_key_match ( ion_flat_file_t flat_file,
ion_flat_file_row_t row,
va_list *  args 
)

Predicate function to return any row that has an exact match to the given target key.

We expect one ion_key_t to be in args.

See also
ion_flat_file_predicate_t

Definition at line 280 of file flat_file.c.

284  {
285  ion_key_t target_key = va_arg(*args, ion_key_t);
286 
287  return ION_FLAT_FILE_STATUS_OCCUPIED == row->row_status && 0 == flat_file->super.compare(target_key, row->key, flat_file->super.record.key_size);
288 }
ion_record_info_t record
void * ion_key_t
A dictionary key.
Definition: kv_system.h:241
ion_flat_file_row_status_t row_status
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
#define ION_FLAT_FILE_STATUS_OCCUPIED
Signifies that this row in the flat file is currently occupied and should not be overwritten.
ion_dictionary_compare_t compare

Here is the caller graph for this function:

ion_boolean_t flat_file_predicate_not_empty ( ion_flat_file_t flat_file,
ion_flat_file_row_t row,
va_list *  args 
)

Predicate function to return any row that is not empty or deleted.

See also
ion_flat_file_predicate_t

Definition at line 268 of file flat_file.c.

272  {
273  UNUSED(flat_file);
274  UNUSED(args);
275 
277 }
ion_flat_file_row_status_t row_status
#define ION_FLAT_FILE_STATUS_OCCUPIED
Signifies that this row in the flat file is currently occupied and should not be overwritten.
#define UNUSED(x)
Definition: kv_system.h:102

Here is the caller graph for this function:

ion_boolean_t flat_file_predicate_within_bounds ( ion_flat_file_t flat_file,
ion_flat_file_row_t row,
va_list *  args 
)

Predicate function to return any row that has a key such that lower_bound <= key <= upper_bound holds true.

We expect two ion_key_t parameters to be in args - the first to represent the lower bound of the key, and the second to represent the upper bound.

See also
ion_flat_file_predicate_t

Definition at line 291 of file flat_file.c.

295  {
296  ion_key_t lower_bound = va_arg(*args, ion_key_t);
297  ion_key_t upper_bound = va_arg(*args, ion_key_t);
298 
299  return ION_FLAT_FILE_STATUS_OCCUPIED == row->row_status && flat_file->super.compare(row->key, lower_bound, flat_file->super.record.key_size) >= 0 && flat_file->super.compare(row->key, upper_bound, flat_file->super.record.key_size) <= 0;
300 }
ion_record_info_t record
void * ion_key_t
A dictionary key.
Definition: kv_system.h:241
ion_flat_file_row_status_t row_status
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
#define ION_FLAT_FILE_STATUS_OCCUPIED
Signifies that this row in the flat file is currently occupied and should not be overwritten.
ion_dictionary_compare_t compare

Here is the caller graph for this function:

ion_err_t flat_file_read_row ( ion_flat_file_t flat_file,
ion_fpos_t  location,
ion_flat_file_row_t row 
)

Reads the row specified by the given location into the buffer.

The returned row is given by attaching pointers correctly from the given row parameter. These pointers are associated with the internal read buffer of the flat file. You should assume that the row does not have a lifetime beyond the scope of where you call this function - any subsequent operation that mutates the read buffer will cause the row to become garbage. Copy the row data out if you want it to persist. If the requested row has already been loaded by a prior call to flat_file_scan, then it will retrieve it as a cache hit directly from the buffer. Otherwise, it will do a seek and read to fetch the row.

Parameters
[in]flat_fileWhich flat file instance to read from.
[in]locationWhich row index to read. This function will compute the file offset of the row index.
[in]rowWrite back row to place read data from the desired location.
Returns
Resulting status of the several file operations used to perform the read.

Definition at line 350 of file flat_file.c.

354  {
355  ion_fpos_t read_index = 0;
356 
357  if ((flat_file->current_loaded_region != -1) && (location >= flat_file->current_loaded_region) && ((unsigned) location < flat_file->current_loaded_region + flat_file->num_in_buffer)) {
358  /* Cache hit, return directly from buffer */
359  read_index = location - flat_file->current_loaded_region;
360  }
361  else {
362  /* Cache miss, have to re-read from file */
363  if (0 != fseek(flat_file->data_file, flat_file->start_of_data + location * flat_file->row_size, SEEK_SET)) {
364  return err_file_bad_seek;
365  }
366 
367  if (1 != fread(flat_file->buffer, sizeof(row->row_status), 1, flat_file->data_file)) {
368  return err_file_write_error;
369  }
370 
371  if (1 != fread(flat_file->buffer + sizeof(row->row_status), flat_file->super.record.key_size, 1, flat_file->data_file)) {
372  return err_file_write_error;
373  }
374 
375  if (1 != fread(flat_file->buffer + sizeof(row->row_status) + flat_file->super.record.key_size, flat_file->super.record.value_size, 1, flat_file->data_file)) {
376  return err_file_write_error;
377  }
378  }
379 
380  row->row_status = *((ion_flat_file_row_status_t *) &flat_file->buffer[read_index * flat_file->row_size]);
381  row->key = &flat_file->buffer[read_index * flat_file->row_size + sizeof(ion_flat_file_row_status_t)];
382  row->value = &flat_file->buffer[read_index * flat_file->row_size + sizeof(ion_flat_file_row_status_t) + flat_file->super.record.key_size];
383 
384  return err_ok;
385 }
ion_record_info_t record
ion_fpos_t current_loaded_region
ion_byte_t ion_flat_file_row_status_t
This type describes the status flag within a flat file row.
ion_byte_t * buffer
ion_flat_file_row_status_t row_status
ion_fpos_t start_of_data
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_value_size_t value_size
Definition: kv_system.h:309

Here is the caller graph for this function:

ion_err_t flat_file_scan ( ion_flat_file_t flat_file,
ion_fpos_t  start_location,
ion_fpos_t location,
ion_flat_file_row_t row,
ion_byte_t  scan_direction,
ion_flat_file_predicate_t  predicate,
  ... 
)

Performs a linear scan of the flat file writing the first location seen that satisfies the given predicate to location.

If the scan falls through, then the location is written as the EOF position of the file. The variadic arguments accepted by this function are passed into the predicate's additional parameters. This can be used to provide additional context to the predicate for use in determining whether or not the row is a match.

Parameters
[in]flat_fileWhich flat file instance to scan.
[in]start_locationWhere to begin the scan. This is given as a row index. If given as -1, then it is assumed to be either the start of file or end of file, depending on the state of scan_direction.
[out]locationAllocated memory location to write back the found location into. Is not changed in the event of a failure or error condition. This location is given back as a row index.
[out]rowA row struct to write back the found row into. This is allocated by the user.
[in]scan_directionScans in the direction provided.
[in]predicateGiven test function to check each row against. Once this function returns true, the scan is terminated and the found location and row are written back to their respective output parameters.
Returns
Resulting status of scan.

Definition at line 157 of file flat_file.c.

165  {
166  ion_fpos_t cur_offset = flat_file->start_of_data + start_location * flat_file->row_size;
167  ion_fpos_t end_offset = ION_FLAT_FILE_SCAN_FORWARDS == scan_direction ? flat_file->eof_position : flat_file->start_of_data;
168 
169  if (-1 == start_location) {
170  if (ION_FLAT_FILE_SCAN_FORWARDS == scan_direction) {
171  cur_offset = flat_file->start_of_data;
172  }
173  else {
174  cur_offset = flat_file->eof_position;
175  }
176  }
177 
178  /* If we're scanning backwards, bump the cur_offset up one record so that we read the record we're sitting on. */
179  /* We don't do this if we're positioned at the EOF, since otherwise we would read garbage. */
180  if ((ION_FLAT_FILE_SCAN_BACKWARDS == scan_direction) && (cur_offset != flat_file->eof_position)) {
181  cur_offset += flat_file->row_size;
182  }
183 
184  if ((cur_offset > flat_file->eof_position) || (cur_offset < flat_file->start_of_data)) {
185  return err_out_of_bounds;
186  }
187 
188  while (cur_offset != end_offset) {
189  if (0 != fseek(flat_file->data_file, cur_offset, SEEK_SET)) {
190  return err_file_bad_seek;
191  }
192 
193  /* We set cur_offset to be the next block to read after this next code segment, so */
194  /* we need t save what block we're currently reading now for location calculation purposes */
195  ion_fpos_t prev_offset = cur_offset;
196  size_t num_records_to_process = flat_file->num_buffered;
197 
198  if (ION_FLAT_FILE_SCAN_FORWARDS == scan_direction) {
199  /* It's possible for this to do a partial read (if you're close to EOF), calculate how many we need to read */
200  size_t records_left = (end_offset - cur_offset) / flat_file->row_size;
201 
202  num_records_to_process = records_left > (unsigned) flat_file->num_buffered ? (unsigned) flat_file->num_buffered : records_left;
203 
204  if (num_records_to_process != fread(flat_file->buffer, flat_file->row_size, num_records_to_process, flat_file->data_file)) {
205  return err_file_read_error;
206  }
207 
208  if (-1 == (cur_offset = ftell(flat_file->data_file))) {
209  return err_file_read_error;
210  }
211  }
212  else {
213  /* Move the offset pointer to the next read location, clamp it at start_of_file if we go too far. */
214  cur_offset -= flat_file->row_size * flat_file->num_buffered;
215 
216  if (cur_offset < flat_file->start_of_data) {
217  /* We know how many rows we went past the start of file, calculate it so we don't fread too much */
218  num_records_to_process = flat_file->num_buffered - (flat_file->start_of_data - cur_offset) / flat_file->row_size;
219  cur_offset = flat_file->start_of_data;
220  }
221 
222  if (0 != fseek(flat_file->data_file, cur_offset, SEEK_SET)) {
223  return err_file_bad_seek;
224  }
225 
226  if (num_records_to_process != fread(flat_file->buffer, flat_file->row_size, num_records_to_process, flat_file->data_file)) {
227  return err_file_read_error;
228  }
229 
230  /* In this case, the prev_offset is actually the cur_offset. */
231  prev_offset = cur_offset;
232  }
233 
234  flat_file->current_loaded_region = (prev_offset - flat_file->start_of_data) / flat_file->row_size;
235  flat_file->num_in_buffer = num_records_to_process;
236 
237  int32_t i;
238 
239  for (i = ION_FLAT_FILE_SCAN_FORWARDS == scan_direction ? 0 : num_records_to_process - 1; ION_FLAT_FILE_SCAN_FORWARDS == scan_direction ? (size_t) i < num_records_to_process : i >= 0; ION_FLAT_FILE_SCAN_FORWARDS == scan_direction ? i++ : i--) {
240  size_t cur_rec = i * flat_file->row_size;
241 
242  /* This cast is done because in the future, the status could possibly be a non-byte type */
243  row->row_status = *((ion_flat_file_row_status_t *) &flat_file->buffer[cur_rec]);
244  row->key = &flat_file->buffer[cur_rec + sizeof(ion_flat_file_row_status_t)];
245  row->value = &flat_file->buffer[cur_rec + sizeof(ion_flat_file_row_status_t) + flat_file->super.record.key_size];
246 
247  va_list predicate_arguments;
248 
249  va_start(predicate_arguments, predicate);
250 
251  ion_boolean_t predicate_test = predicate(flat_file, row, &predicate_arguments);
252 
253  va_end(predicate_arguments);
254 
255  if (predicate_test) {
256  *location = (prev_offset - flat_file->start_of_data) / flat_file->row_size + i;
257  return err_ok;
258  }
259  }
260  }
261 
262  /* If we reach this point, then no row matched the predicate. */
263  *location = (flat_file->eof_position - flat_file->start_of_data) / flat_file->row_size;
264  return err_file_hit_eof;
265 }
ion_dictionary_size_t num_buffered
ion_record_info_t record
ion_fpos_t current_loaded_region
ion_byte_t ion_flat_file_row_status_t
This type describes the status flag within a flat file row.
#define ION_FLAT_FILE_SCAN_BACKWARDS
Signals to flat_file_scan to scan in a backward direction.
ion_byte_t * buffer
ion_flat_file_row_status_t row_status
ion_fpos_t start_of_data
ion_fpos_t eof_position
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
A supertype for cursor predicate objects.
#define ION_FLAT_FILE_SCAN_FORWARDS
Signals to flat_file_scan to scan in a forward direction.
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
char ion_boolean_t
A boolean type.
Definition: kv_system.h:269

Here is the caller graph for this function:

ion_status_t flat_file_update ( ion_flat_file_t flat_file,
ion_key_t  key,
ion_value_t  value 
)

Updates all records stored with the given key to have value.

Parameters
[in]flat_fileWhich flat file to update in.
[in]keySpecified key to find and update.
[in]valueNew value to replace old values with.
Returns
Resulting status of the operation.
See also
ffdict_update

Definition at line 560 of file flat_file.c.

564  {
566 
567  ion_fpos_t loc = -1;
569  ion_err_t err;
570 
571  if (flat_file->sorted_mode) {
572  err = flat_file_binary_search(flat_file, key, &loc);
573 
574  if (err_ok != err) {
575  if (err_item_not_found == err) {
576  /* Key didn't exist, do upsert. This may fail because it violates the sorted order. */
577  return flat_file_insert(flat_file, key, value);
578  }
579 
580  status.error = err;
581  return status;
582  }
583 
584  err = flat_file_read_row(flat_file, loc, &row);
585 
586  if (err_ok != err) {
587  status.error = err;
588  return status;
589  }
590 
591  if (0 != flat_file->super.compare(row.key, key, flat_file->super.record.key_size)) {
592  /* Key didn't exist, do upsert. */
593  return flat_file_insert(flat_file, key, value);
594  }
595  }
596 
597  while (err_ok == (err = flat_file_scan(flat_file, loc, &loc, &row, ION_FLAT_FILE_SCAN_FORWARDS, flat_file_predicate_key_match, key))) {
598  ion_err_t row_err = flat_file_write_row(flat_file, loc, &(ion_flat_file_row_t) { ION_FLAT_FILE_STATUS_OCCUPIED, key, value });
599 
600  if (err_ok != row_err) {
601  status.error = row_err;
602  return status;
603  }
604 
605  status.count++;
606  /* Move one-forwards to skip the one we just updated */
607  loc++;
608  }
609 
610  status.error = err_ok;
611 
612  if ((err == err_file_hit_eof) && (status.count == 0)) {
613  /* If this is the case, then we had nothing to update. Do an upsert instead */
614  return flat_file_insert(flat_file, key, value);
615  }
616  else if (err != err_file_hit_eof) {
617  status.error = err;
618  }
619 
620  return status;
621 }
ion_err_t flat_file_write_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Writes the given row out to the data file.
Definition: flat_file.c:321
ion_err_t flat_file_binary_search(ion_flat_file_t *flat_file, ion_key_t target_key, ion_fpos_t *location)
Performs a binary search for the given target_key, returning to location the first-less-than-or-equal...
Definition: flat_file.c:638
ion_record_info_t record
Container for the rows written in the flat file data file.
#define key(k)
Definition: bpp_tree.c:75
ion_err_t error
Definition: kv_system.h:291
char ion_err_t
The error type used to store error codes.
Definition: kv_system.h:226
ion_err_t flat_file_read_row(ion_flat_file_t *flat_file, ion_fpos_t location, ion_flat_file_row_t *row)
Reads the row specified by the given location into the buffer.
Definition: flat_file.c:350
ion_err_t flat_file_scan(ion_flat_file_t *flat_file, ion_fpos_t start_location, ion_fpos_t *location, ion_flat_file_row_t *row, ion_byte_t scan_direction, ion_flat_file_predicate_t predicate,...)
Performs a linear scan of the flat file writing the first location seen that satisfies the given pred...
Definition: flat_file.c:157
ion_result_count_t count
Definition: kv_system.h:293
ion_dictionary_parent_t super
ion_key_size_t key_size
Definition: kv_system.h:307
#define ION_FLAT_FILE_STATUS_OCCUPIED
Signifies that this row in the flat file is currently occupied and should not be overwritten.
#define ION_FLAT_FILE_SCAN_FORWARDS
Signals to flat_file_scan to scan in a forward direction.
ion_boolean_t sorted_mode
ion_boolean_t flat_file_predicate_key_match(ion_flat_file_t *flat_file, ion_flat_file_row_t *row, va_list *args)
Predicate function to return any row that has an exact match to the given target key.
Definition: flat_file.c:280
long ion_fpos_t
A file position type.
Definition: kv_system.h:237
ion_dictionary_compare_t compare
ion_status_t flat_file_insert(ion_flat_file_t *flat_file, ion_key_t key, ion_value_t value)
Inserts the given record into the flat file store.
Definition: flat_file.c:388
#define ION_STATUS_INITIALIZE
Definition: kv_system.h:107
A status object that describes the result of a dictionary operation.
Definition: kv_system.h:290

Here is the call graph for this function:

Here is the caller graph for this function: