Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NC | Online Upgrade | Tests | Config directory restructure upgrade script unit tests #8654

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

NC | Online Upgrade | Tests | Config directory restructure upgrade script unit tests #8654

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 86 additions & 16 deletions src/test/system_tests/test_utils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -482,15 +482,15 @@ function get_new_buckets_path_by_test_env(new_buckets_full_path, new_buckets_dir
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {Object} config_data
* @param {String} [invalid_str]
* @param {{symlink_name?: Boolean, symlink_access_key?: Boolean}} [options]
* @returns {Promise<Void>}
*/
async function write_manual_config_file(type, config_fs, config_data, invalid_str = '') {
async function write_manual_config_file(type, config_fs, config_data, invalid_str = '', { symlink_name, symlink_access_key} = {symlink_name: true, symlink_access_key: true}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, in this approach you used default values for the whole object, so only if someone doesn't pass the options argument would it get those defaults.
Otherwise, if one passes one of them, it would get only the one that he passed.
Is that what you meant?

const config_path = type === CONFIG_TYPES.BUCKET ?
config_fs.get_bucket_path_by_name(config_data.name) :
config_fs.get_identity_path_by_id(config_data._id);
if (type === CONFIG_TYPES.ACCOUNT) {
const dir_path = config_fs.get_identity_dir_path_by_id(config_data._id);
await nb_native().fs.mkdir(config_fs.fs_context, dir_path, native_fs_utils.get_umasked_mode(config.BASE_MODE_DIR));
await create_identity_dir_if_missing(config_fs, config_data);
}
await nb_native().fs.writeFile(
config_fs.fs_context,
Expand All @@ -500,22 +500,62 @@ async function write_manual_config_file(type, config_fs, config_data, invalid_st
mode: native_fs_utils.get_umasked_mode(config.BASE_MODE_FILE)
}
);
const id_relative_path = config_fs.get_account_relative_path_by_id(config_data._id);

if (type === CONFIG_TYPES.ACCOUNT) {
const id_relative_path = config_fs.get_account_relative_path_by_id(config_data._id);
const name_symlink_path = config_fs.get_account_or_user_path_by_name(config_data.name);
await nb_native().fs.symlink(config_fs.fs_context, id_relative_path, name_symlink_path);
if (type === CONFIG_TYPES.ACCOUNT && symlink_name) {
await symlink_account_name(config_fs, config_data.name, id_relative_path);
}

if (type === CONFIG_TYPES.ACCOUNT && symlink_access_key && config_data.access_keys) {
await symlink_account_access_key(config_fs, config_data.access_keys[0].access_key, id_relative_path);
Comment on lines +509 to +510
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I'm not sure it is enough && config_data.access_keys - anonymous account would have access_key as [] (defined, but empty array).
  2. If you can change it from the first item (config_data.access_keys[0]) to iterate on the array of access_keys, as this is a helper function and we might have more than 1 access key.

}
}

/**
* symlink_account_name symlinks the account's name path to the target link path
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {String} account_name
* @param {String} link_target
*/
async function symlink_account_name(config_fs, account_name, link_target) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add in the JSDoc when we will use it?

const name_symlink_path = config_fs.get_account_or_user_path_by_name(account_name);
await nb_native().fs.symlink(config_fs.fs_context, link_target, name_symlink_path);
}

/**
* symlink_account_access_key symlinks the account's access key path to the target link path
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {String} access_key
* @param {String} link_target
*/
async function symlink_account_access_key(config_fs, access_key, link_target) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add in the JSDoc when we will use it?

const access_key_symlink_path = config_fs.get_account_or_user_path_by_access_key(access_key);
await nb_native().fs.symlink(config_fs.fs_context, link_target, access_key_symlink_path);
}

/**
* create_identity_dir_if_missing created the identity directory if missing
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {Object} config_data
* @returns {Promise<Void>}
*/
async function create_identity_dir_if_missing(config_fs, config_data) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can pass only the ID as an argument (instead of the config_data object).

const dir_path = config_fs.get_identity_dir_path_by_id(config_data._id);
try {
await nb_native().fs.mkdir(config_fs.fs_context, dir_path, native_fs_utils.get_umasked_mode(config.BASE_MODE_DIR));
} catch (err) {
if (err.code !== 'ENOENT') throw err;
}
}

/**
* write_manual_old_account_config_file writes account config file directly to the old file system account path without using config FS
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {Object} config_data
* @param {{symlink_access_key?: Boolean}} [options]
* @returns {Promise<Void>}
*/
async function write_manual_old_account_config_file(config_fs, config_data) {
async function write_manual_old_account_config_file(config_fs, config_data, { symlink_access_key } = { symlink_access_key: false }) {
const config_path = config_fs._get_old_account_path_by_name(config_data.name);
await nb_native().fs.writeFile(
config_fs.fs_context,
Expand All @@ -525,6 +565,12 @@ async function write_manual_old_account_config_file(config_fs, config_data) {
mode: native_fs_utils.get_umasked_mode(config.BASE_MODE_FILE)
}
);

const account_name_relative_path = config_fs.get_old_account_relative_path_by_name(config_data.name);
if (symlink_access_key) {
const access_key_symlink_path = config_fs.get_account_or_user_path_by_access_key(config_data.access_keys[0].access_key);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(copied comment) - If you can change it from the first item (config_data.access_keys[0]) to iterate on the array of access_keys, as this is a helper function and we might have more than 1 access key.

await nb_native().fs.symlink(config_fs.fs_context, account_name_relative_path, access_key_symlink_path);
}
}

/**
Expand Down Expand Up @@ -556,9 +602,9 @@ async function delete_manual_config_file(type, config_fs, config_data) {

/**
* @param {any} test_name
* @param {Object} [config_fs]
* @param {import('../../sdk/config_fs').ConfigFS} [config_fs]
*/
async function fail_test_if_default_config_dir_exists(test_name, config_fs = {}) {
async function fail_test_if_default_config_dir_exists(test_name, config_fs) {
const fs_context = config_fs?.fs_context || native_fs_utils.get_process_fs_context();
const config_dir_exists = await native_fs_utils.is_path_exists(fs_context, config.NSFS_NC_DEFAULT_CONF_DIR);
const msg = `${test_name} found an existing default config directory ${config.NSFS_NC_DEFAULT_CONF_DIR},` +
Expand All @@ -583,6 +629,8 @@ async function create_config_dir(config_dir) {

/**
* clean_config_dir cleans the config directory
* @param {import('../../sdk/config_fs').ConfigFS} config_fs
* @param {String} [custom_config_dir_path]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add more details in the JSDoc about custom_config_dir_path?

* @returns {Promise<Void>}
*/
async function clean_config_dir(config_fs, custom_config_dir_path) {
Expand All @@ -593,17 +641,35 @@ async function clean_config_dir(config_fs, custom_config_dir_path) {
const system_json = '/system.json';
romayalon marked this conversation as resolved.
Show resolved Hide resolved
for (const dir of [buckets_dir_name, identities_dir_name, access_keys_dir_name, accounts_by_name, config.NSFS_TEMP_CONF_DIR_NAME]) {
const default_path = path.join(config.NSFS_NC_DEFAULT_CONF_DIR, dir);
await fs_utils.folder_delete(default_path);
const custom_path = path.join(custom_config_dir_path, dir);
await fs_utils.folder_delete(custom_path);

await fs_utils.folder_delete_skip_enoent(default_path);
if (custom_config_dir_path) {
const custom_path = path.join(custom_config_dir_path, dir);
await fs_utils.folder_delete_skip_enoent(custom_path);
}
}

await delete_redirect_file(config_fs);
await fs_utils.file_delete(system_json);
await fs_utils.folder_delete(config.NSFS_NC_DEFAULT_CONF_DIR);
await fs_utils.folder_delete(custom_config_dir_path);
await fs_utils.folder_delete_skip_enoent(config.NSFS_NC_DEFAULT_CONF_DIR);
await fs_utils.folder_delete_skip_enoent(custom_config_dir_path);
}

/**
* create_file creates a file in the file system
* @param {nb.NativeFSContext} fs_context
* @param {String} file_path
* @param {Object} file_data
*/
async function create_file(fs_context, file_path, file_data) {
await nb_native().fs.writeFile(
fs_context,
file_path,
Buffer.from(JSON.stringify(file_data)),
{
mode: native_fs_utils.get_umasked_mode(config.BASE_MODE_FILE)
}
);
}

exports.blocks_exist_on_cloud = blocks_exist_on_cloud;
exports.create_hosts_pool = create_hosts_pool;
Expand All @@ -629,6 +695,10 @@ exports.get_new_buckets_path_by_test_env = get_new_buckets_path_by_test_env;
exports.write_manual_config_file = write_manual_config_file;
exports.write_manual_old_account_config_file = write_manual_old_account_config_file;
exports.delete_manual_config_file = delete_manual_config_file;
exports.create_identity_dir_if_missing = create_identity_dir_if_missing;
exports.symlink_account_name = symlink_account_name;
exports.symlink_account_access_key = symlink_account_access_key;
exports.create_file = create_file;
exports.create_redirect_file = create_redirect_file;
exports.delete_redirect_file = delete_redirect_file;
exports.fail_test_if_default_config_dir_exists = fail_test_if_default_config_dir_exists;
Expand Down
Loading
Loading