Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add plugin-copy-column #611

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 141 additions & 0 deletions plugins/copy-column/src/copy-column.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
import api, { Flatfile } from "@flatfile/api"
import type { FlatfileEvent, FlatfileListener } from '@flatfile/listener'
import { updateRecords, Simplified } from "../../../utils/common/src"

export type Primitive = string | number | null | boolean
export type SimpleRecord = Record<string, Primitive>

export function copyColumn() {
return (listener: FlatfileListener) => {
listener.on('workbook:created', async ({ context: { fileId, workbookId } }: FlatfileEvent) => {
const { data: sheets } = await api.sheets.list({ workbookId })

sheets.forEach(async (sheet) => {
const loadAllFields = await getAllFields(sheets, sheet.slug);
setSheetActions(sheet, [
copyColumnValuesBlueprint(loadAllFields),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat to populate all the field enums for each sheet!
Does this workbook:created event get triggered for File based Workbooks? Would we want to add the Action on every workbook:created or sheet:created?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not seeing sheet:created when creating a space. What do you mean by File based Workbooks?

...(sheet.config.actions?.filter(
(a) =>
a.operation !== 'copy-column'
) || []),
]);
})
Comment on lines +13 to +22
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use 'for...of' loop instead of 'forEach' with async callbacks

Using Array.forEach with an async callback does not wait for the promises to resolve, which can lead to unexpected behavior. It's recommended to use a for...of loop to handle asynchronous operations sequentially.

Apply this diff to fix the issue:

- sheets.forEach(async (sheet) => {
+ for (const sheet of sheets) {
+   const loadAllFields = await getAllFields(sheets, sheet.slug);
+   await setSheetActions(sheet, [
+     copyColumnValuesBlueprint(loadAllFields),
+     ...(sheet.config.actions?.filter(
+       (a) =>
+         a.operation !== 'copy-column'
+     ) || []),
+   ]);
+ }
-   const loadAllFields = await getAllFields(sheets, sheet.slug);
-   setSheetActions(sheet, [
-     copyColumnValuesBlueprint(loadAllFields),
-     ...(sheet.config.actions?.filter(
-       (a) =>
-         a.operation !== 'copy-column'
-     ) || []),
-   ]);
- })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
sheets.forEach(async (sheet) => {
const loadAllFields = await getAllFields(sheets, sheet.slug);
setSheetActions(sheet, [
copyColumnValuesBlueprint(loadAllFields),
...(sheet.config.actions?.filter(
(a) =>
a.operation !== 'copy-column'
) || []),
]);
})
for (const sheet of sheets) {
const loadAllFields = await getAllFields(sheets, sheet.slug);
await setSheetActions(sheet, [
copyColumnValuesBlueprint(loadAllFields),
...(sheet.config.actions?.filter(
(a) =>
a.operation !== 'copy-column'
) || []),
]);
}

})

listener.on("job:ready", { job: "sheet:copy-column" }, async (e) => {
const { records } = await e.data({ pageLength: 10_000 })
const { data: job } = await api.jobs.get(e.context.jobId)
const { key_copy_from, key_paste_to } = job.input
const patch = records.map(copyColumnValues(key_copy_from, key_paste_to))

Comment on lines +28 to +30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Validate input keys before copying column values

There is no validation to ensure that key_copy_from and key_paste_to are valid and not the same. Adding validation will prevent potential errors and unintended data overwrites.

Apply this diff to add validation:

  const { key_copy_from, key_paste_to } = job.input
+ if (!key_copy_from || !key_paste_to || key_copy_from === key_paste_to) {
+   throw new Error('Invalid input: Source and destination columns must be specified and cannot be the same.')
+ }
  const patch = records.map(copyColumnValues(key_copy_from, key_paste_to))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const { key_copy_from, key_paste_to } = job.input
const patch = records.map(copyColumnValues(key_copy_from, key_paste_to))
const { key_copy_from, key_paste_to } = job.input
if (!key_copy_from || !key_paste_to || key_copy_from === key_paste_to) {
throw new Error('Invalid input: Source and destination columns must be specified and cannot be the same.')
}
const patch = records.map(copyColumnValues(key_copy_from, key_paste_to))

await updateRecords(e.context.sheetId, patch)

await api.jobs.complete(e.context.jobId, {
outcome: {
message: `Data was successfully copied from ${key_copy_from} to ${key_paste_to}.`,
},
});
})
}
}

function copyColumnValues(from_key: string, to_key: string) {
return (record: Flatfile.Record_) => {
const obj = Simplified.toSimpleRecord(record)
return {
id: record.id,
values: {
[to_key]: { value: obj[from_key] },
flatfile-nullify[bot] marked this conversation as resolved.
Show resolved Hide resolved
},
}
}
}

/**
* Set sheet actions
*
* @param sheet
* @param actions
*/
async function setSheetActions(sheet: any, actions: Flatfile.Action[]) {
try {
const { constraints } = sheet.config;
await api.workbooks.update(sheet.workbookId, {
sheets: [
{
name: sheet.name,
slug: sheet.config.slug,
actions,
},
],
});

if (constraints?.length > 0) {
await api.workbooks.update(sheet.workbookId, {
sheets: [
{
...sheet,
config: {
...sheet.config,
actions,
constraints,
},
},
],
});
}
} catch (e) {
console.log({ e });
}
Comment on lines +88 to +89
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Enhance error handling in the 'setSheetActions' function

Currently, the error is logged using console.log({ e });, which might not be sufficient for debugging and monitoring. Consider implementing a more robust error handling mechanism, such as rethrowing the error or logging it using a dedicated logging service.

Apply this diff to improve error handling:

  } catch (e) {
-   console.log({ e });
+   console.error('Error updating sheet actions:', e);
+   throw e;
  }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
console.log({ e });
}
console.error('Error updating sheet actions:', e);
throw e;
}

}

export const getAllFields = (sheets, sheetSlug) => {
const _fields = sheets.reduce((acc, sheet) => {
console.log("in getallfields, sheet.config.fields here: ")
console.log(sheet.config)
if (sheet.config.slug == sheetSlug) {
acc.push(
...sheet.config.fields
.map((f) => {
return { ...f, appearance: { size: "s" } } as const
})
)
}
return acc
}, [] as Flatfile.Property[])

return _fields
}

export const copyColumnValuesBlueprint = (columns: Flatfile.Property[]): Flatfile.Action => ({
label: "Copy Column",
primary: false,
operation: "copy-column",
inputForm: {
type: "simple",
fields: [
{
key: "key_copy_from",
type: "enum",
label: "Copy data from column",
config: {
options: columns
.map((f) => ({ label: f.label, value: f.key }))
.sort((a, b) => a.label.localeCompare(b.label)),
},
constraints: [{ type: "required" }],
},
{
key: "key_paste_to",
type: "enum",
label: "Paste data to column",
config: {
options: columns
.map((f) => ({ label: f.label, value: f.key }))
.sort((a, b) => a.label.localeCompare(b.label)),
},
constraints: [{ type: "required" }],
},
],
},
})
1 change: 1 addition & 0 deletions plugins/copy-column/src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './copy-column'
Loading