Experimental Specifications

mmmvxspec_1: Project API Specification (is DEPRECATED)



This specification is DEPRECATED.

Situation Description, Observations and some Ideas


The algorihtm for transforming a symlink-less an_element_E1 dependency tree to the state that is described in the Diagram 1 is that the build system of the an_element_E1 receives a hash-table. The data format of the hash-table is that its

keys are package names

and the

values are full paths to a symlink or a folder that either contains or references the packages .

In the case of the Diagram 1 the hash-table that the build system of the an_element_E1 receives is empty, but it might contain one key-value pair, where the key is the dep_X1 and the value is the full path to the sandbox version of the dep_X1 at the machine of the dev_A. The build system of the an_element_E1 compares the list of an_element_E1 immediate dependencies, which is dep_X1, dep_42, dep_22, with the list in the hash-table and replaces non-recursively those immediate dependencies that are listed in the hash-table, with symlinks to the paths that are listed in the hash-table. The replacement is carried out even if the immediate dependency is in a form of a symlink before the replacement. Then the build system of the an_element_E1 creates a new, empty, hash-table, lists its own immediate dependencies with the full paths that reside in the project tree of the an_element_E1 and adds to that hash-table only those key-value pairs from the received hash-table that do not depict the immediate dependencies of the an_element_E1. The build system passes the newly created hash-table to the build systems of all of those immediate dependencies of the an_element_E1 that were not listed in the hash-table that the build system of the an_element_E1 received and the whole process is repeated recursively.

In the case of this example the algorithm leaves one redundant copy of the dep_UFO_1 to the project tree of the an_element_E1. That inefficiency is eliminated by creating a package, optimization_hack_1, that has the an_element_E1 as its immediate dependency. Then the optimization_hack_1 dependencies that have more than one folder, regardless of whether they are remote or immediate dependencies of the optimization_hack_1, are copied to be immediate dependencies of the optimization_hack_1. After the copying the previously described symlink creation algorithm is run. The optimized result for the case at the Diagram 1:




The API

The general idea is to use the Decorator Design Pattern so that the Project API calls recurse down the dependency tree. To avoid dependency version collisions in Project API implementations, the Project API calls are run in separate operating system processes. The API is console/bash/shell based.

some_application_possibly_rake_or_bashscript command_name command_parameters

where in some cases the implementation is simplified by combining command_name and command_parameters to separate, new, commands. For example, if there were a parametrized command "move", then in stead of

move left
move right
move backwards


the command "move" would be replaced with 3 separate commands that do not take any parameters:

move_left
move_right
move_backwards


In the case of the Project API one of the command parameters determines, whether the result is printed to console in human readable form or in some structured text format, like JSON, YAML, ProgFTE, etc.

Proposed Project API in EBNF:

API ::= API_RUNNER COMMAND
API_RUNNER ::= Rakefile | bashscript
COMMAND ::= COMMAND_FOR_HUMANREADABLE_OUTPUT | COMMAND_FOR_STRUCTURED_TEXT_OUTPUT

COMMAND_FOR_HUMANREADABLE_OUTPUT ::= COMMANDS_H_1 | COMMANDS_H_2
COMMANDS_H_1 ::= mmmvxspec_1_optimize_storage_recursively_t1 FILE_PATH_2? | mmmvxspec_1_unoptimize_storage_recursively_t1 FILE_PATH_2?
COMMANDS_H_2 ::= mmmvxspec_1_recreate_project_specific_symlinks_recursively_t1

COMMAND_FOR_STRUCTURED_TEXT_OUTPUT ::= COMMANDS_TX_1 | COMMANDS_TX_2 | COMMANDS_TX_3 | COMMANDS_TX_4 | COMMANDS_TX_5
COMMANDS_TX_1 ::= mmmvxspec_1_optimize_storage_recursively_t1_progfte FILE_PATH_2? | mmmvxspec_1_unoptimize_storage_recursively_t1_progfte FILE_PATH_2?
COMMANDS_TX_2 ::= mmmvxspec_1_recreate_project_specific_symlinks_recursively_t1_progfte
COMMANDS_TX_3 ::= mmmvxspec_1_folder2symlink_t1_without_recursion_progfte FILE_PATH_1 | mmmvxspec_1_symlink2folder_t1_without_recursion_progfte FILE_PATH_1
COMMANDS_TX_4 ::= mmmvxspec_1_get_list_of_immediate_dependencies_t1_progfte | mmmvxspec_1_get_list_of_folders_of_dependencies_recursively_t1_progfte
COMMANDS_TX_5 ::= mmmvxspec_1_convert_relative_symlinks_2_absolute_symlinks_recursively_t1_progfte | mmmvxspec_1_convert_absolute_symlinks_2_relative_symlinks_recursively_t1_progfte


The FILE_PATH_1 is a full path to an "upstream" version

The FILE_PATH_2 is a full path to a temporary text-file that contains ProgFTE of the following hash-table:

ht
x+--[<name of the folder or symlink that holds the dependency in project lib folder >] => < full path to the "upstream" location of the folder or symlink>


The mmmvxspec_1_get_list_of_immediate_dependencies_t1_progfte includes both, symlinks and folders, to the list prints the list to console as ProgFTE text that uses the same hash-table data format as is used in the file that is referenced by the FILE_PATH_2.

The mmmvxspec_1_get_list_of_folders_of_dependencies_recursively_t1_progfte does not include symlinks to the list and returns its results by printing ProgFTE to console. The result of the mmmvxspec_1_get_list_of_folders_of_dependencies_recursively_t1_progfte has the following hash-table data format:

ht
x+--[<name of the folder or symlink that holds the dependency in project lib folder >] => < ProgFTE of a hash-table that lists full paths by placing them at keys "0", "1", "2", ...>


API_RUNNER Path

The API_RUNNER is executed by changing the working directory to the path of the API_RUNNER. The API_RUNNER is searched according to the following precedence:

  1. $PROJECT_ROOT/src/dev_tools/Rakefile

  2. $PROJECT_ROOT/bonnet/project_api/mmmvxspec_1/apirunner.bash

  3. $PROJECT_ROOT/src/bonnet/project_api/mmmvxspec_1/apirunner.bash

  4. Throw an exception and stop. All packages that are listed as an immediate dependency of any other package are expected to have an API_RUNNER.



Additional Notes and Comments

According to the current scheme different versions of the a same software project are expected to be represented as different packages. That allows substantial API changes within the package and it allows the different versions to be simultaneously present in the tree of dependencies. Characters that have special meaning in Bash are not allowed to be part of package names. That includes spaces, semicolons, parenthesis, etc.

If the API_RUNNER implementation creates the symlinks with absolute paths, then the symlinks will probabilistically become dead whenever the optimization_hack_1 is copied from one computer to another. One way to "fix" that might be to re-run the symlink creation routine on optimization_hack_1 before starting to use the optimization_hack_1, but that will not work if some of the API_RUNNER dependencies are referenced by the very same, dead, symlinks that the API_RUNNER is expected to overwrite. Therefore the symlinks that reference packages within the optimization_hack_1 must be relative symlinks, because that way the API_RUNNER should, hopefully, stay functional. As relative symlinks probabilistically break, if their location is changed relative to their target, the relative symlinks must be converted to absolute symlinks prior to the switch between folders and symlinks. In step by step fashion:

mmmvxspec_1_optimize_storage_recursively_t1
  1. xxxxconvert relative symlinks to absolute symlinks
  2. xxxxapply the recursive folders 2 symlinks conversion algorithm
  3. xxxxconvert absolute symlinks to relative symlinks


mmmvxspec_1_unoptimize_storage_recursively_t1
  1. xxxxconvert relative symlinks to absolute symlinks
  2. xxxxcopy folders to replace symlinks till no copying takes place
  3. xxxxdo nothing, because there are no absolute symlinks left to convert to relative symlinks




Some Related References