Future_implementation.txt

April 22, 2021
- Data check file to perform checks on data values such as NaN, reasonable value check, missing values, 
  format of the file as expected. 
- "Flow from directory" manual implementation that is similar to TensorFlow's image's "flow from directory".
  This might need optimization, how to combine each packet on the par into single csv file, is this the most
  optimized method? Is there any faster way (perhaps with TensorFlow Text)? How will we flow the correct
  corresponding file from both params and noisy? 
- Transformation of data by preprocess it first and store it permanently separately so it does not have to 
  be preprocessed every time it loads. The decision on a decided preprocessing technique requires discussion
  with Rachel. 

Downsides currently: 
- All methods of reading is an extreme bottleneck. Flow from directory does not flow as smoothly as we wanted
  it to perhaps because of the loop operations and concatenation purposes.