-
Notifications
You must be signed in to change notification settings - Fork 1
Training with Kraken
Stefan Weil edited this page Aug 9, 2022
·
3 revisions
Trained models are available from https://ub-backup.bib.uni-mannheim.de/~stweil/tesstrain/kraken/digitue-gt/.
The training is based on an existing model, see https://github.com/UB-Mannheim/reichsanzeiger-gt/wiki/Training-with-Kraken.
# Get all images.
for xml in */*/*.xml */*/*/*.xml; do dir=$(dirname $xml); img=$(grep imageFilename $xml | sed s/.*imageFilename=.// | sed s/jpg.*/jpg/); url=$(grep externalRef $xml | sed s/.*externalRef=.// | sed s/jpg.*/jpg/); echo $img $url; (cd $dir && curl -sS -o $img $url); done
nice ketos train -f page -i ../reichsanzeiger-gt/reichsanzeiger_best.mlmodel -o digitue -d cuda:0 --lag 10 -r 0.0001 -B 1 -w 0 --resize add */*/*.xml */*/*/*.xml
WARNING:root:Torch version 1.13.0.dev20220803+cu113 has not been tested with coremltools. You may run into unexpected errors. Torch 1.10.2 is the most recent version that has been tested.
[08/08/22 07:46:19] WARNING Text line "" is empty after transformations train.py:351
WARNING Text line "" is empty after transformations train.py:351
WARNING Text line "" is empty after transformations train.py:351
[08/08/22 07:46:20] WARNING Text line "" is empty after transformations train.py:351
WARNING Text line "" is empty after transformations train.py:351
WARNING Text line "" is empty after transformations train.py:351
Trainer already configured with model summary callbacks: [<class 'pytorch_lightning.callbacks.rich_model_summary.RichModelSummary'>]. Skipping setting a default `ModelSummary` callback.
GPU available: True, used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
`Trainer(val_check_interval=1.0)` was configured so validation will run at the end of the training epoch..
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
┏━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓
┃ ┃ Name ┃ Type ┃ Params ┃
┡━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩
│ 0 │ net │ MultiParamSequential │ 4.1 M │
│ 1 │ net.C_0 │ ActConv2D │ 1.3 K │
│ 2 │ net.Do_1 │ Dropout │ 0 │
│ 3 │ net.Mp_2 │ MaxPool │ 0 │
│ 4 │ net.C_3 │ ActConv2D │ 40.0 K │
│ 5 │ net.Do_4 │ Dropout │ 0 │
│ 6 │ net.Mp_5 │ MaxPool │ 0 │
│ 7 │ net.C_6 │ ActConv2D │ 55.4 K │
│ 8 │ net.Do_7 │ Dropout │ 0 │
│ 9 │ net.Mp_8 │ MaxPool │ 0 │
│ 10 │ net.C_9 │ ActConv2D │ 110 K │
│ 11 │ net.Do_10 │ Dropout │ 0 │
│ 12 │ net.S_11 │ Reshape │ 0 │
│ 13 │ net.L_12 │ TransposedSummarizingRNN │ 1.9 M │
│ 14 │ net.Do_13 │ Dropout │ 0 │
│ 15 │ net.L_14 │ TransposedSummarizingRNN │ 963 K │
│ 16 │ net.Do_15 │ Dropout │ 0 │
│ 17 │ net.L_16 │ TransposedSummarizingRNN │ 963 K │
│ 18 │ net.Do_17 │ Dropout │ 0 │
│ 19 │ net.O_18 │ LinSoftmax │ 100 K │
└────┴───────────┴──────────────────────────┴────────┘
Trainable params: 4.1 M
Non-trainable params: 0
Total params: 4.1 M
Total estimated model params size (MB): 16
stage 0/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:05 val_accuracy: 0.98590 early_stopping: 0/10 0.98590
stage 1/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:11 val_accuracy: 0.99465 early_stopping: 0/10 0.99465
stage 2/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:20 val_accuracy: 0.99571 early_stopping: 0/10 0.99571
stage 3/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:07 val_accuracy: 0.99647 early_stopping: 0/10 0.99647
stage 4/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:21:57 val_accuracy: 0.99681 early_stopping: 0/10 0.99681
stage 5/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:03 val_accuracy: 0.99737 early_stopping: 0/10 0.99737
stage 6/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:21 val_accuracy: 0.99730 early_stopping: 1/10 0.99737
stage 7/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:14 val_accuracy: 0.99739 early_stopping: 0/10 0.99739
stage 8/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:12 val_accuracy: 0.99733 early_stopping: 1/10 0.99739
stage 9/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:06 val_accuracy: 0.99743 early_stopping: 0/10 0.99743
stage 10/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:29 val_accuracy: 0.99739 early_stopping: 1/10 0.99743
stage 11/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:53 val_accuracy: 0.99748 early_stopping: 0/10 0.99748
stage 12/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:24:02 val_accuracy: 0.99755 early_stopping: 0/10 0.99755
stage 13/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:23 val_accuracy: 0.99771 early_stopping: 0/10 0.99771
stage 14/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:37 val_accuracy: 0.99786 early_stopping: 0/10 0.99786
stage 15/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:27 val_accuracy: 0.99793 early_stopping: 0/10 0.99793
stage 16/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:34 val_accuracy: 0.99777 early_stopping: 1/10 0.99793
stage 17/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:24 val_accuracy: 0.99775 early_stopping: 2/10 0.99793
stage 18/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:19 val_accuracy: 0.99766 early_stopping: 3/10 0.99793
stage 19/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:18 val_accuracy: 0.99773 early_stopping: 4/10 0.99793
stage 20/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:37 val_accuracy: 0.99762 early_stopping: 5/10 0.99793
stage 21/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:45 val_accuracy: 0.99789 early_stopping: 6/10 0.99793
stage 22/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:23 val_accuracy: 0.99768 early_stopping: 7/10 0.99793
stage 23/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:35 val_accuracy: 0.99784 early_stopping: 8/10 0.99793
stage 24/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:40 val_accuracy: 0.99789 early_stopping: 9/10 0.99793
stage 25/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:43 val_accuracy: 0.99802 early_stopping: 0/10 0.99802
stage 26/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:25:24 val_accuracy: 0.99807 early_stopping: 0/10 0.99807
stage 27/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:24:24 val_accuracy: 0.99800 early_stopping: 1/10 0.99807
stage 28/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:51 val_accuracy: 0.99827 early_stopping: 0/10 0.99827
stage 29/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:05 val_accuracy: 0.99818 early_stopping: 1/10 0.99827
stage 30/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:15 val_accuracy: 0.99802 early_stopping: 2/10 0.99827
stage 31/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:06 val_accuracy: 0.99800 early_stopping: 3/10 0.99827
stage 32/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:10 val_accuracy: 0.99813 early_stopping: 4/10 0.99827
stage 33/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:43 val_accuracy: 0.99804 early_stopping: 5/10 0.99827
stage 34/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:42 val_accuracy: 0.99827 early_stopping: 6/10 0.99827
stage 35/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:55 val_accuracy: 0.99816 early_stopping: 7/10 0.99827
stage 36/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:22:54 val_accuracy: 0.99823 early_stopping: 8/10 0.99827
stage 37/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:07 val_accuracy: 0.99822 early_stopping: 9/10 0.99827
stage 38/∞ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7071/7071 0:00:00 0:23:43 val_accuracy: 0.99818 early_stopping: 10/10 0.99827
Moving best model digitue_28.mlmodel (0.9982700943946838) to digitue_best.mlmodel
real 902m9.851s
user 2222m27.741s
sys 5300m35.515s