03/24/2023, 21:42:46: Namespace(seed=2022, data_dir='data/', logging_dir='log/sports/pretrain_t5_small_sports_random.log', model_dir='model/sports/pretrain_t5_small_sports_random.pt', task='sports', max_history=20, sequential_num=10, negative_sample=2, yes_no_sample=5, direct_item_proportion=2, train_sequential_item_batch=64, train_sequential_yesno_batch=32, train_direct_yesno_batch=48, train_direct_candidate_batch=12, train_direct_straightforward_batch=48, meta_title_batch=128, meta_description_batch=12, meta_category_batch=128, meta_epochs=10, meta_lr=0.0001, review_user_batch=16, review_item_batch=16, review_epochs=2, review_lr=0.0001, model_type='t5-small', epochs=20, lr=0.0005, clip=1.0, logging_step=1000, warmup_prop=0.05, gradient_accumulation_steps=1, weight_decay=0.01, adam_eps=1e-06, dropout=0.1, alpha=2, multiGPU=True, distributed=True, gpu='2,3', local_rank=0, evaluation_method='sequential_item', evaluation_template_id=0, item_representation='random_number', cluster_number=55, cluster_size=100, optimal_width_in_CF=False, data_order='random', remapped_data_order='original', resolution=2, max_random_number=22000, min_random_number=1000, overlap=0, base=10, random_initialization_embedding=True, whole_word_embedding='shijie', no_pretrain=False, use_meta_data=False, use_review_data=False, remove_last_item=False, remove_first_item=False, eval_only=False, check_data=False)

03/24/2023, 21:43:22: Process Launching at GPU 1

03/24/2023, 21:43:22: Process Launching at GPU 0

03/24/2023, 21:43:23: Building train loader at GPU 0

03/24/2023, 21:43:23: loading data ...

03/24/2023, 21:43:23: Building train loader at GPU 1

03/24/2023, 21:43:24: finished loading data

03/24/2023, 21:43:24: length of training data is 14809

03/24/2023, 21:43:24: loading model ...

03/24/2023, 21:43:24: using only sequential data, and all possible sequences are here

03/24/2023, 21:43:24: use pretrained T5

03/24/2023, 21:43:26: randomly initialize number-related embeddings only

03/24/2023, 21:43:32: Batch per epoch: 14809

03/24/2023, 21:43:32: Total steps: 296180

03/24/2023, 21:43:32: Warmup proportion:

03/24/2023, 21:43:32: Warm up steps: 14809

03/24/2023, 21:43:32: model dataparallel set

03/24/2023, 21:43:32: start training

03/24/2023, 21:43:32: ---------- training epoch 0 ----------

03/24/2023, 21:45:57: total loss for 1000 steps : 4342.386345863342

03/24/2023, 21:48:19: total loss for 2000 steps : 2464.514574766159

03/24/2023, 21:50:41: total loss for 3000 steps : 2323.750068426132

03/24/2023, 21:53:03: total loss for 4000 steps : 2235.99568772316

03/24/2023, 21:55:25: total loss for 5000 steps : 2175.0636744499207

03/24/2023, 21:57:47: total loss for 6000 steps : 2122.969354391098

03/24/2023, 22:00:11: total loss for 7000 steps : 2083.996894836426

03/24/2023, 22:02:35: total loss for 8000 steps : 2054.571484684944

03/24/2023, 22:04:59: total loss for 9000 steps : 2032.2053635120392

03/24/2023, 22:07:24: total loss for 10000 steps : 2011.3294196128845

03/24/2023, 22:09:48: total loss for 11000 steps : 1994.0636135339737

03/24/2023, 22:12:13: total loss for 12000 steps : 1978.3188925981522

03/24/2023, 22:14:34: total loss for 13000 steps : 1964.057801604271

03/24/2023, 22:16:56: total loss for 14000 steps : 1953.4433690309525

03/24/2023, 22:18:51: ---------- start evaluation after epoch 0 ----------

03/24/2023, 22:29:01: validation hit @ 1 is 0.0020225855385133996

03/24/2023, 22:29:01: validation hit @ 5 is 0.005281195572784988

03/24/2023, 22:29:01: validation hit @ 10 is 0.007528512837799877

03/24/2023, 22:29:01: validation ncdg @ 5 is 0.003943801038949487

03/24/2023, 22:29:01: validation ncdg @ 10 is 0.003943801038949487

03/24/2023, 22:29:08: validation hit @ 1 is 0.002415866059891005

03/24/2023, 22:29:08: validation hit @ 5 is 0.006573403000168549

03/24/2023, 22:29:08: validation hit @ 10 is 0.009045451991684926

03/24/2023, 22:29:08: validation ncdg @ 5 is 0.004667287050825397

03/24/2023, 22:29:08: validation ncdg @ 10 is 0.004667287050825397

03/24/2023, 22:39:23: test hit @ 1 is 0.001573122085510422

03/24/2023, 22:39:23: test hit @ 5 is 0.005225012641159616

03/24/2023, 22:39:23: test hit @ 10 is 0.007022866453171526

03/24/2023, 22:39:23: test ncdg @ 5 is 0.0036704743762502275

03/24/2023, 22:39:23: test ncdg @ 10 is 0.0036704743762502275

03/24/2023, 22:39:23: recall increases from 0 ----> 0.007528512837799877 at epoch 0

03/24/2023, 22:39:23: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/24/2023, 22:39:32: test hit @ 1 is 0.001854036743637283

03/24/2023, 22:39:32: test hit @ 5 is 0.005281195572784988

03/24/2023, 22:39:32: test hit @ 10 is 0.007528512837799877

03/24/2023, 22:39:32: test ncdg @ 5 is 0.003757178991805049

03/24/2023, 22:39:32: test ncdg @ 10 is 0.003757178991805049

03/24/2023, 22:39:32: recall increases from 0 ----> 0.009045451991684926 at epoch 0

03/24/2023, 22:39:32: ---------- training epoch 1 ----------

03/24/2023, 22:40:01: total loss for 15000 steps : 1944.55235517025

03/24/2023, 22:42:29: total loss for 16000 steps : 1936.780630350113

03/24/2023, 22:44:58: total loss for 17000 steps : 1928.139416694641

03/24/2023, 22:47:30: total loss for 18000 steps : 1921.2898797988892

03/24/2023, 22:50:00: total loss for 19000 steps : 1917.6609094142914

03/24/2023, 22:52:32: total loss for 20000 steps : 1911.353964805603

03/24/2023, 22:55:03: total loss for 21000 steps : 1904.8478090763092

03/24/2023, 22:57:34: total loss for 22000 steps : 1902.0638101100922

03/24/2023, 23:00:05: total loss for 23000 steps : 1902.9693298339844

03/24/2023, 23:02:33: total loss for 24000 steps : 1896.0824033021927

03/24/2023, 23:05:00: total loss for 25000 steps : 1895.8773529529572

03/24/2023, 23:07:27: total loss for 26000 steps : 1888.4894618988037

03/24/2023, 23:09:53: total loss for 27000 steps : 1889.1168401241302

03/24/2023, 23:12:20: total loss for 28000 steps : 1889.174058675766

03/24/2023, 23:14:48: total loss for 29000 steps : 1884.9731122255325

03/24/2023, 23:16:21: ---------- start evaluation after epoch 1 ----------

03/24/2023, 23:26:33: validation hit @ 1 is 0.0020225855385133996

03/24/2023, 23:26:33: validation hit @ 5 is 0.006236305410416316

03/24/2023, 23:26:33: validation hit @ 10 is 0.008876903196808809

03/24/2023, 23:26:33: validation ncdg @ 5 is 0.0043712282925922824

03/24/2023, 23:26:33: validation ncdg @ 10 is 0.0043712282925922824

03/24/2023, 23:26:43: validation hit @ 1 is 0.002415866059891005

03/24/2023, 23:26:43: validation hit @ 5 is 0.007528512837799877

03/24/2023, 23:26:43: validation hit @ 10 is 0.010337659419068487

03/24/2023, 23:26:43: validation ncdg @ 5 is 0.0051908747153988745

03/24/2023, 23:26:43: validation ncdg @ 10 is 0.0051908747153988745

03/24/2023, 23:36:41: test hit @ 1 is 0.001573122085510422

03/24/2023, 23:36:41: test hit @ 5 is 0.005449744367661105

03/24/2023, 23:36:41: test hit @ 10 is 0.00786561042755211

03/24/2023, 23:36:41: test ncdg @ 5 is 0.0037540831867191185

03/24/2023, 23:36:41: test ncdg @ 10 is 0.0037540831867191185

03/24/2023, 23:36:41: recall increases from 0.007528512837799877 ----> 0.008876903196808809 at epoch 1

03/24/2023, 23:36:41: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/24/2023, 23:36:57: test hit @ 1 is 0.001854036743637283

03/24/2023, 23:36:57: test hit @ 5 is 0.005449744367661105

03/24/2023, 23:36:57: test hit @ 10 is 0.006910500589920782

03/24/2023, 23:36:57: test ncdg @ 5 is 0.0038347237232418145

03/24/2023, 23:36:57: test ncdg @ 10 is 0.0038347237232418145

03/24/2023, 23:36:57: recall increases from 0.009045451991684926 ----> 0.010337659419068487 at epoch 1

03/24/2023, 23:36:57: ---------- training epoch 2 ----------

03/24/2023, 23:37:53: total loss for 30000 steps : 1882.3179886341095

03/24/2023, 23:40:19: total loss for 31000 steps : 1877.6173074245453

03/24/2023, 23:42:45: total loss for 32000 steps : 1876.0388811826706

03/24/2023, 23:45:19: total loss for 33000 steps : 1874.711900472641

03/24/2023, 23:47:48: total loss for 34000 steps : 1871.2870582342148

03/24/2023, 23:50:19: total loss for 35000 steps : 1864.3074612617493

03/24/2023, 23:52:50: total loss for 36000 steps : 1866.3101915121078

03/24/2023, 23:55:19: total loss for 37000 steps : 1863.3838843107224

03/24/2023, 23:57:50: total loss for 38000 steps : 1857.5847593545914

03/25/2023, 00:00:22: total loss for 39000 steps : 1854.4840737581253

03/25/2023, 00:02:52: total loss for 40000 steps : 1852.1346229314804

03/25/2023, 00:05:22: total loss for 41000 steps : 1848.981766819954

03/25/2023, 00:07:52: total loss for 42000 steps : 1841.6924302577972

03/25/2023, 00:10:22: total loss for 43000 steps : 1839.676627755165

03/25/2023, 00:12:51: total loss for 44000 steps : 1833.2637083530426

03/25/2023, 00:13:55: ---------- start evaluation after epoch 2 ----------

03/25/2023, 00:24:16: validation hit @ 1 is 0.0038766222821506827

03/25/2023, 00:24:16: validation hit @ 5 is 0.011910781504578909

03/25/2023, 00:24:16: validation hit @ 10 is 0.01780998932524299

03/25/2023, 00:24:16: validation ncdg @ 5 is 0.008026007145350036

03/25/2023, 00:24:16: validation ncdg @ 10 is 0.008026007145350036

03/25/2023, 00:24:29: validation hit @ 1 is 0.003988988145401427

03/25/2023, 00:24:29: validation hit @ 5 is 0.012079330299455026

03/25/2023, 00:24:29: validation hit @ 10 is 0.01724816000898927

03/25/2023, 00:24:29: validation ncdg @ 5 is 0.008096809337084076

03/25/2023, 00:24:29: validation ncdg @ 10 is 0.008096809337084076

03/25/2023, 00:34:27: test hit @ 1 is 0.002359683128265633

03/25/2023, 00:34:27: test hit @ 5 is 0.007640878701050621

03/25/2023, 00:34:27: test hit @ 10 is 0.011011854598572953

03/25/2023, 00:34:27: test ncdg @ 5 is 0.005004838278385966

03/25/2023, 00:34:27: test ncdg @ 10 is 0.005004838278385966

03/25/2023, 00:34:27: recall increases from 0.010337659419068487 ----> 0.01780998932524299 at epoch 2

03/25/2023, 00:34:36: test hit @ 1 is 0.0030338783077700996

03/25/2023, 00:34:36: test hit @ 5 is 0.008652171470307321

03/25/2023, 00:34:36: test hit @ 10 is 0.012360244957581886

03/25/2023, 00:34:36: test ncdg @ 5 is 0.005912624975512184

03/25/2023, 00:34:36: test ncdg @ 10 is 0.005912624975512184

03/25/2023, 00:34:36: recall increases from 0.008876903196808809 ----> 0.01724816000898927 at epoch 2

03/25/2023, 00:34:36: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/25/2023, 00:34:37: ---------- training epoch 3 ----------

03/25/2023, 00:36:00: total loss for 45000 steps : 1830.9957062005997

03/25/2023, 00:38:24: total loss for 46000 steps : 1824.1309351921082

03/25/2023, 00:40:50: total loss for 47000 steps : 1819.7404440641403

03/25/2023, 00:43:18: total loss for 48000 steps : 1815.3188457489014

03/25/2023, 00:45:51: total loss for 49000 steps : 1809.6691719293594

03/25/2023, 00:48:23: total loss for 50000 steps : 1802.4271544218063

03/25/2023, 00:50:59: total loss for 51000 steps : 1798.6071251630783

03/25/2023, 00:53:31: total loss for 52000 steps : 1795.8887205123901

03/25/2023, 00:56:03: total loss for 53000 steps : 1790.8257060050964

03/25/2023, 00:58:35: total loss for 54000 steps : 1785.6055761575699

03/25/2023, 01:01:08: total loss for 55000 steps : 1779.536811709404

03/25/2023, 01:03:39: total loss for 56000 steps : 1776.9070893526077

03/25/2023, 01:06:09: total loss for 57000 steps : 1771.8027565479279

03/25/2023, 01:08:40: total loss for 58000 steps : 1766.47911465168

03/25/2023, 01:11:09: total loss for 59000 steps : 1762.621323108673

03/25/2023, 01:11:43: ---------- start evaluation after epoch 3 ----------

03/25/2023, 01:21:53: validation hit @ 1 is 0.006741951795044665

03/25/2023, 01:21:53: validation hit @ 5 is 0.017922355188493735

03/25/2023, 01:21:53: validation hit @ 10 is 0.02511377043654138

03/25/2023, 01:21:53: validation ncdg @ 5 is 0.012297233626262836

03/25/2023, 01:21:53: validation ncdg @ 10 is 0.012297233626262836

03/25/2023, 01:22:01: validation hit @ 1 is 0.008427439743805831

03/25/2023, 01:22:01: validation hit @ 5 is 0.020225855385133995

03/25/2023, 01:22:01: validation hit @ 10 is 0.027866734086184616

03/25/2023, 01:22:01: validation ncdg @ 5 is 0.014278519399974687

03/25/2023, 01:22:01: validation ncdg @ 10 is 0.014278519399974687

03/25/2023, 01:32:12: test hit @ 1 is 0.004944097983032755

03/25/2023, 01:32:12: test hit @ 5 is 0.012360244957581886

03/25/2023, 01:32:12: test hit @ 10 is 0.018877465026125063

03/25/2023, 01:32:12: test ncdg @ 5 is 0.008658032444614303

03/25/2023, 01:32:12: test ncdg @ 10 is 0.008658032444614303

03/25/2023, 01:32:12: recall increases from 0.01724816000898927 ----> 0.02511377043654138 at epoch 3

03/25/2023, 01:32:12: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/25/2023, 01:32:27: test hit @ 1 is 0.005056463846283499

03/25/2023, 01:32:27: test hit @ 5 is 0.012247879094331142

03/25/2023, 01:32:27: test hit @ 10 is 0.017697623461992246

03/25/2023, 01:32:27: test ncdg @ 5 is 0.008670048886816906

03/25/2023, 01:32:27: test ncdg @ 10 is 0.008670048886816906

03/25/2023, 01:32:27: recall increases from 0.01780998932524299 ----> 0.027866734086184616 at epoch 3

03/25/2023, 01:32:27: ---------- training epoch 4 ----------

03/25/2023, 01:34:20: total loss for 60000 steps : 1756.2727026939392

03/25/2023, 01:36:47: total loss for 61000 steps : 1748.983936548233

03/25/2023, 01:39:13: total loss for 62000 steps : 1741.8070253133774

03/25/2023, 01:41:40: total loss for 63000 steps : 1739.8943247795105

03/25/2023, 01:44:07: total loss for 64000 steps : 1733.4402517080307

03/25/2023, 01:46:38: total loss for 65000 steps : 1726.8005208969116

03/25/2023, 01:49:09: total loss for 66000 steps : 1728.8655529022217

03/25/2023, 01:51:40: total loss for 67000 steps : 1717.8400801420212

03/25/2023, 01:54:09: total loss for 68000 steps : 1714.6142429113388

03/25/2023, 01:56:39: total loss for 69000 steps : 1708.6978524923325

03/25/2023, 01:59:07: total loss for 70000 steps : 1703.552996635437

03/25/2023, 02:01:37: total loss for 71000 steps : 1699.9269669055939

03/25/2023, 02:04:04: total loss for 72000 steps : 1692.9258776903152

03/25/2023, 02:06:33: total loss for 73000 steps : 1686.9605906009674

03/25/2023, 02:09:02: total loss for 74000 steps : 1683.5433502197266

03/25/2023, 02:09:08: ---------- start evaluation after epoch 4 ----------

03/25/2023, 02:19:37: validation hit @ 1 is 0.012753525478959492

03/25/2023, 02:19:37: validation hit @ 5 is 0.027304904769930895

03/25/2023, 02:19:37: validation hit @ 10 is 0.036687454351368054

03/25/2023, 02:19:37: validation ncdg @ 5 is 0.02019615599074731

03/25/2023, 02:19:37: validation ncdg @ 10 is 0.02019615599074731

03/25/2023, 02:19:47: validation hit @ 1 is 0.010843305803696838

03/25/2023, 02:19:47: validation hit @ 5 is 0.02410247766728468

03/25/2023, 02:19:47: validation hit @ 10 is 0.031855722231586045

03/25/2023, 02:19:47: validation ncdg @ 5 is 0.017493470539723516

03/25/2023, 02:19:47: validation ncdg @ 10 is 0.017493470539723516

03/25/2023, 02:30:04: test hit @ 1 is 0.007640878701050621

03/25/2023, 02:30:04: test hit @ 5 is 0.01679869655598629

03/25/2023, 02:30:04: test hit @ 10 is 0.0236530142142817

03/25/2023, 02:30:04: test ncdg @ 5 is 0.012413595946830067

03/25/2023, 02:30:04: test ncdg @ 10 is 0.012413595946830067

03/25/2023, 02:30:04: recall increases from 0.027866734086184616 ----> 0.036687454351368054 at epoch 4

03/25/2023, 02:30:28: test hit @ 1 is 0.008202708017304343

03/25/2023, 02:30:28: test hit @ 5 is 0.017416708803865387

03/25/2023, 02:30:28: test hit @ 10 is 0.02337209955615484

03/25/2023, 02:30:28: test ncdg @ 5 is 0.012973134248325396

03/25/2023, 02:30:28: test ncdg @ 10 is 0.012973134248325396

03/25/2023, 02:30:28: recall increases from 0.02511377043654138 ----> 0.031855722231586045 at epoch 4

03/25/2023, 02:30:28: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/25/2023, 02:30:29: ---------- training epoch 5 ----------

03/25/2023, 02:32:52: total loss for 75000 steps : 1674.6406440734863

03/25/2023, 02:35:18: total loss for 76000 steps : 1668.416299700737

03/25/2023, 02:37:43: total loss for 77000 steps : 1660.6169329881668

03/25/2023, 02:40:08: total loss for 78000 steps : 1654.8442220687866

03/25/2023, 02:42:34: total loss for 79000 steps : 1650.8982260227203

03/25/2023, 02:44:59: total loss for 80000 steps : 1644.2791041135788

03/25/2023, 02:47:29: total loss for 81000 steps : 1638.2261908054352

03/25/2023, 02:50:01: total loss for 82000 steps : 1633.458182811737

03/25/2023, 02:52:33: total loss for 83000 steps : 1624.0392169952393

03/25/2023, 02:55:06: total loss for 84000 steps : 1618.0813829898834

03/25/2023, 02:57:37: total loss for 85000 steps : 1611.9221395254135

03/25/2023, 03:00:10: total loss for 86000 steps : 1604.2826071977615

03/25/2023, 03:02:42: total loss for 87000 steps : 1601.5103524923325

03/25/2023, 03:05:12: total loss for 88000 steps : 1592.1488811969757

03/25/2023, 03:07:19: ---------- start evaluation after epoch 5 ----------

03/25/2023, 03:17:49: validation hit @ 1 is 0.014944659812349008

03/25/2023, 03:17:49: validation hit @ 5 is 0.033428844317096466

03/25/2023, 03:17:49: validation hit @ 10 is 0.04393505253104107

03/25/2023, 03:17:49: validation ncdg @ 5 is 0.024266689158436404

03/25/2023, 03:17:49: validation ncdg @ 10 is 0.024266689158436404

03/25/2023, 03:17:58: validation hit @ 1 is 0.010843305803696838

03/25/2023, 03:17:58: validation hit @ 5 is 0.02511377043654138

03/25/2023, 03:17:58: validation hit @ 10 is 0.032136636889712904

03/25/2023, 03:17:58: validation ncdg @ 5 is 0.0182418930237367

03/25/2023, 03:17:58: validation ncdg @ 10 is 0.0182418930237367

03/25/2023, 03:28:09: test hit @ 1 is 0.00786561042755211

03/25/2023, 03:28:09: test hit @ 5 is 0.018203269846620598

03/25/2023, 03:28:09: test hit @ 10 is 0.024832855778414518

03/25/2023, 03:28:09: test ncdg @ 5 is 0.013296420341729223

03/25/2023, 03:28:09: test ncdg @ 10 is 0.013296420341729223

03/25/2023, 03:28:09: recall increases from 0.031855722231586045 ----> 0.032136636889712904 at epoch 5

03/25/2023, 03:28:09: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/25/2023, 03:28:10: test hit @ 1 is 0.010225293555817742

03/25/2023, 03:28:10: test hit @ 5 is 0.02112478229113995

03/25/2023, 03:28:10: test hit @ 10 is 0.030338783077700996

03/25/2023, 03:28:10: test ncdg @ 5 is 0.015625213242798846

03/25/2023, 03:28:10: test ncdg @ 10 is 0.015625213242798846

03/25/2023, 03:28:10: recall increases from 0.036687454351368054 ----> 0.04393505253104107 at epoch 5

03/25/2023, 03:28:31: ---------- training epoch 6 ----------

03/25/2023, 03:28:53: total loss for 89000 steps : 1585.002724647522

03/25/2023, 03:31:22: total loss for 90000 steps : 1576.4039680957794

03/25/2023, 03:33:53: total loss for 91000 steps : 1568.840106010437

03/25/2023, 03:36:21: total loss for 92000 steps : 1560.8699523210526

03/25/2023, 03:38:47: total loss for 93000 steps : 1553.7040548324585

03/25/2023, 03:41:13: total loss for 94000 steps : 1549.143539428711

03/25/2023, 03:43:40: total loss for 95000 steps : 1541.4780304431915

03/25/2023, 03:46:09: total loss for 96000 steps : 1533.0491490364075

03/25/2023, 03:48:39: total loss for 97000 steps : 1526.3293064832687

03/25/2023, 03:51:11: total loss for 98000 steps : 1517.9770814180374

03/25/2023, 03:53:42: total loss for 99000 steps : 1508.8764992952347

03/25/2023, 03:56:14: total loss for 100000 steps : 1503.2098566293716

03/25/2023, 03:58:46: total loss for 101000 steps : 1495.0930107831955

03/25/2023, 04:01:18: total loss for 102000 steps : 1484.7608337402344

03/25/2023, 04:03:50: total loss for 103000 steps : 1480.3920401334763

03/25/2023, 04:05:29: ---------- start evaluation after epoch 6 ----------

03/25/2023, 04:15:56: validation hit @ 1 is 0.016180684308107197

03/25/2023, 04:15:56: validation hit @ 5 is 0.036237990898365074

03/25/2023, 04:15:56: validation hit @ 10 is 0.04685656497556043

03/25/2023, 04:15:56: validation ncdg @ 5 is 0.026506805874544105

03/25/2023, 04:15:56: validation ncdg @ 10 is 0.026506805874544105

03/25/2023, 04:16:06: validation hit @ 1 is 0.01039384235069386

03/25/2023, 04:16:06: validation hit @ 5 is 0.025563233889544356

03/25/2023, 04:16:06: validation hit @ 10 is 0.033428844317096466

03/25/2023, 04:16:06: validation ncdg @ 5 is 0.01830300318445744

03/25/2023, 04:16:06: validation ncdg @ 10 is 0.01830300318445744

03/25/2023, 04:26:27: test hit @ 1 is 0.010618574077195348

03/25/2023, 04:26:27: test hit @ 5 is 0.022192257992022024

03/25/2023, 04:26:27: test hit @ 10 is 0.031574807573459186

03/25/2023, 04:26:27: test ncdg @ 5 is 0.016558491523333187

03/25/2023, 04:26:27: test ncdg @ 10 is 0.016558491523333187

03/25/2023, 04:26:27: recall increases from 0.04393505253104107 ----> 0.04685656497556043 at epoch 6

03/25/2023, 04:26:56: test hit @ 1 is 0.008202708017304343

03/25/2023, 04:26:56: test hit @ 5 is 0.01932692847912804

03/25/2023, 04:26:56: test hit @ 10 is 0.026012697342547336

03/25/2023, 04:26:56: test ncdg @ 5 is 0.013963500256524408

03/25/2023, 04:26:56: test ncdg @ 10 is 0.013963500256524408

03/25/2023, 04:26:56: recall increases from 0.032136636889712904 ----> 0.033428844317096466 at epoch 6

03/25/2023, 04:26:56: save current best model to best_model/sports/pretrain_t5_small_sports_random.pt

03/25/2023, 04:26:57: ---------- training epoch 7 ----------

03/25/2023, 04:27:48: total loss for 104000 steps : 1473.277843952179

03/25/2023, 04:30:17: total loss for 105000 steps : 1454.8309993743896

03/25/2023, 04:32:46: total loss for 106000 steps : 1452.380856871605

03/25/2023, 04:35:15: total loss for 107000 steps : 1442.7143124341965

03/25/2023, 04:37:42: total loss for 108000 steps : 1433.230413198471

03/25/2023, 04:40:10: total loss for 109000 steps : 1424.835225701332

03/25/2023, 04:42:38: total loss for 110000 steps : 1421.4071024656296

03/25/2023, 04:45:02: total loss for 111000 steps : 1411.945384502411

03/25/2023, 04:47:30: total loss for 112000 steps : 1400.9228175878525

03/25/2023, 04:50:00: total loss for 113000 steps : 1392.8605363368988

03/25/2023, 04:52:31: total loss for 114000 steps : 1383.968964099884

03/25/2023, 04:55:03: total loss for 115000 steps : 1377.9885244369507

03/25/2023, 04:57:34: total loss for 116000 steps : 1370.482095837593

03/25/2023, 05:00:07: total loss for 117000 steps : 1361.2729048728943

03/25/2023, 05:02:39: total loss for 118000 steps : 1350.5821710824966

03/25/2023, 05:03:51: ---------- start evaluation after epoch 7 ----------

03/25/2023, 05:14:22: validation hit @ 1 is 0.01623686723973257

03/25/2023, 05:14:22: validation hit @ 5 is 0.03427158829147705

03/25/2023, 05:14:22: validation hit @ 10 is 0.04567672341142761

03/25/2023, 05:14:22: validation ncdg @ 5 is 0.025483158770403833

03/25/2023, 05:14:22: validation ncdg @ 10 is 0.025483158770403833

03/25/2023, 05:14:36: validation hit @ 1 is 0.010843305803696838

03/25/2023, 05:14:36: validation hit @ 5 is 0.024158660598910052

03/25/2023, 05:14:36: validation hit @ 10 is 0.031855722231586045

03/25/2023, 05:14:36: validation ncdg @ 5 is 0.01765216204862593

03/25/2023, 05:14:36: validation ncdg @ 10 is 0.01765216204862593

03/25/2023, 05:24:55: test hit @ 1 is 0.00938254958143716

03/25/2023, 05:24:55: test hit @ 5 is 0.021180965222765323

03/25/2023, 05:24:55: test hit @ 10 is 0.030394966009326365

03/25/2023, 05:24:55: test ncdg @ 5 is 0.01531082033694621

03/25/2023, 05:24:55: test ncdg @ 10 is 0.01531082033694621

03/25/2023, 05:25:25: test hit @ 1 is 0.008034159222428226

03/25/2023, 05:25:25: test hit @ 5 is 0.018090903983369853

03/25/2023, 05:25:25: test hit @ 10 is 0.023990111804033935

03/25/2023, 05:25:25: test ncdg @ 5 is 0.013145376206500078

03/25/2023, 05:25:25: test ncdg @ 10 is 0.013145376206500078

03/25/2023, 05:25:25: ---------- training epoch 8 ----------

03/25/2023, 05:26:44: total loss for 119000 steps : 1341.0107786655426

03/25/2023, 05:29:11: total loss for 120000 steps : 1328.7577220201492

03/25/2023, 05:31:38: total loss for 121000 steps : 1321.431956768036

03/25/2023, 05:34:06: total loss for 122000 steps : 1313.640725016594

03/25/2023, 05:36:33: total loss for 123000 steps : 1307.399264574051

03/25/2023, 05:38:59: total loss for 124000 steps : 1301.1794840097427

03/25/2023, 05:41:26: total loss for 125000 steps : 1293.2828150987625

03/25/2023, 05:43:55: total loss for 126000 steps : 1281.816881775856

03/25/2023, 05:46:23: total loss for 127000 steps : 1277.2939645051956

03/25/2023, 05:48:56: total loss for 128000 steps : 1263.4463782310486

03/25/2023, 05:51:30: total loss for 129000 steps : 1255.0815937519073

03/25/2023, 05:54:04: total loss for 130000 steps : 1244.524447441101

03/25/2023, 05:56:37: total loss for 131000 steps : 1237.4515216350555

03/25/2023, 05:59:10: total loss for 132000 steps : 1228.9807314872742

03/25/2023, 06:01:45: total loss for 133000 steps : 1223.7905102968216

03/25/2023, 06:02:27: ---------- start evaluation after epoch 8 ----------

03/25/2023, 06:12:59: validation hit @ 1 is 0.015057025675599753

03/25/2023, 06:12:59: validation hit @ 5 is 0.03444013708635316

03/25/2023, 06:12:59: validation hit @ 10 is 0.04567672341142761

03/25/2023, 06:12:59: validation ncdg @ 5 is 0.025051739576490477

03/25/2023, 06:12:59: validation ncdg @ 10 is 0.025051739576490477

03/25/2023, 06:13:06: validation hit @ 1 is 0.009101634923310299

03/25/2023, 06:13:06: validation hit @ 5 is 0.021293331086016068

03/25/2023, 06:13:06: validation hit @ 10 is 0.029889319624698017

03/25/2023, 06:13:06: validation ncdg @ 5 is 0.015427911950118037

03/25/2023, 06:13:06: validation ncdg @ 10 is 0.015427911950118037

03/25/2023, 06:23:23: test hit @ 1 is 0.008708354401932693

03/25/2023, 06:23:23: test hit @ 5 is 0.021405696949266813

03/25/2023, 06:23:23: test hit @ 10 is 0.03000168548794876

03/25/2023, 06:23:23: test ncdg @ 5 is 0.015196832019338953

03/25/2023, 06:23:23: test ncdg @ 10 is 0.015196832019338953

03/25/2023, 06:23:40: test hit @ 1 is 0.005955390752289454

03/25/2023, 06:23:40: test hit @ 5 is 0.014888476880723636

03/25/2023, 06:23:40: test hit @ 10 is 0.021293331086016068

03/25/2023, 06:23:40: test ncdg @ 5 is 0.010596810591054764

03/25/2023, 06:23:40: test ncdg @ 10 is 0.010596810591054764

03/25/2023, 06:23:40: ---------- training epoch 9 ----------

03/25/2023, 06:25:28: total loss for 134000 steps : 1210.9688792228699

03/25/2023, 06:27:57: total loss for 135000 steps : 1200.634392619133

03/25/2023, 06:30:26: total loss for 136000 steps : 1194.873879790306

03/25/2023, 06:32:56: total loss for 137000 steps : 1186.625120997429

03/25/2023, 06:35:25: total loss for 138000 steps : 1179.2768660187721

03/25/2023, 06:37:52: total loss for 139000 steps : 1169.7566499114037

03/25/2023, 06:40:19: total loss for 140000 steps : 1164.2382853031158

03/25/2023, 06:42:46: total loss for 141000 steps : 1154.4909794330597

03/25/2023, 06:45:13: total loss for 142000 steps : 1144.527915596962

03/25/2023, 06:47:41: total loss for 143000 steps : 1134.7125450372696

03/25/2023, 06:50:13: total loss for 144000 steps : 1130.4497433900833

03/25/2023, 06:52:44: total loss for 145000 steps : 1125.0656831860542

03/25/2023, 06:55:14: total loss for 146000 steps : 1115.5537186861038

03/25/2023, 06:57:44: total loss for 147000 steps : 1106.3606078624725

03/25/2023, 07:00:15: total loss for 148000 steps : 1099.8809769749641

03/25/2023, 07:00:30: ---------- start evaluation after epoch 9 ----------

03/25/2023, 07:11:00: validation hit @ 1 is 0.014776111017472891

03/25/2023, 07:11:00: validation hit @ 5 is 0.03314792965896961

03/25/2023, 07:11:00: validation hit @ 10 is 0.04415978425754256

03/25/2023, 07:11:00: validation ncdg @ 5 is 0.024435000629986034

03/25/2023, 07:11:00: validation ncdg @ 10 is 0.024435000629986034

03/25/2023, 07:11:08: validation hit @ 1 is 0.00814652508567897

03/25/2023, 07:11:08: validation hit @ 5 is 0.02011348952188325

03/25/2023, 07:11:08: validation hit @ 10 is 0.02842856340243834

03/25/2023, 07:11:08: validation ncdg @ 5 is 0.014297764404292652

03/25/2023, 07:11:08: validation ncdg @ 10 is 0.014297764404292652

03/25/2023, 07:21:34: test hit @ 1 is 0.008034159222428226

03/25/2023, 07:21:34: test hit @ 5 is 0.01983257486375639

03/25/2023, 07:21:34: test hit @ 10 is 0.02876566099219057

03/25/2023, 07:21:34: test ncdg @ 5 is 0.014057360764254609

03/25/2023, 07:21:34: test ncdg @ 10 is 0.014057360764254609

03/25/2023, 07:21:48: test hit @ 1 is 0.005899207820664082

03/25/2023, 07:21:48: test hit @ 5 is 0.013764818248216192

03/25/2023, 07:21:48: test hit @ 10 is 0.019944940727007136

03/25/2023, 07:21:48: test ncdg @ 5 is 0.009861303619930317

03/25/2023, 07:21:48: test ncdg @ 10 is 0.009861303619930317

03/25/2023, 07:21:48: ---------- training epoch 10 ----------

03/25/2023, 07:24:04: total loss for 149000 steps : 1087.6117621064186

03/25/2023, 07:26:32: total loss for 150000 steps : 1081.677501320839

03/25/2023, 07:29:00: total loss for 151000 steps : 1072.8101263046265

03/25/2023, 07:31:28: total loss for 152000 steps : 1065.6386193037033

03/25/2023, 07:33:57: total loss for 153000 steps : 1058.6027824878693

03/25/2023, 07:36:23: total loss for 154000 steps : 1050.573745727539

03/25/2023, 07:38:47: total loss for 155000 steps : 1042.518130004406

03/25/2023, 07:41:12: total loss for 156000 steps : 1036.3497647047043

03/25/2023, 07:43:37: total loss for 157000 steps : 1025.8690609931946

03/25/2023, 07:46:02: total loss for 158000 steps : 1017.3884484171867

03/25/2023, 07:48:28: total loss for 159000 steps : 1014.7902075648308

03/25/2023, 07:50:59: total loss for 160000 steps : 999.6013796329498

03/25/2023, 07:53:30: total loss for 161000 steps : 998.0892350077629

03/25/2023, 07:56:02: total loss for 162000 steps : 987.3586660027504

03/25/2023, 07:58:21: ---------- start evaluation after epoch 10 ----------

03/25/2023, 08:09:01: validation hit @ 1 is 0.012865891342210237

03/25/2023, 08:09:01: validation hit @ 5 is 0.03118152705208158

03/25/2023, 08:09:01: validation hit @ 10 is 0.04174391819765155

03/25/2023, 08:09:01: validation ncdg @ 5 is 0.02209001176569662

03/25/2023, 08:09:01: validation ncdg @ 10 is 0.02209001176569662

03/25/2023, 08:09:06: validation hit @ 1 is 0.007079049384796899

03/25/2023, 08:09:06: validation hit @ 5 is 0.017753806393617618

03/25/2023, 08:09:06: validation hit @ 10 is 0.024720489915163773

03/25/2023, 08:09:06: validation ncdg @ 5 is 0.012550500745772807

03/25/2023, 08:09:06: validation ncdg @ 10 is 0.012550500745772807

03/25/2023, 08:19:32: test hit @ 1 is 0.007191415248047643

03/25/2023, 08:19:32: test hit @ 5 is 0.017697623461992246

03/25/2023, 08:19:32: test hit @ 10 is 0.02618124613742345

03/25/2023, 08:19:32: test ncdg @ 5 is 0.01245533794348927

03/25/2023, 08:19:32: test ncdg @ 10 is 0.01245533794348927

03/25/2023, 08:19:46: test hit @ 1 is 0.004494634530029777

03/25/2023, 08:19:46: test hit @ 5 is 0.011517500983201303

03/25/2023, 08:19:46: test hit @ 10 is 0.016742513624360918

03/25/2023, 08:19:46: test ncdg @ 5 is 0.007947157221789687

03/25/2023, 08:19:46: test ncdg @ 10 is 0.007947157221789687

03/25/2023, 08:19:46: ---------- training epoch 11 ----------

03/25/2023, 08:20:01: total loss for 163000 steps : 980.3110293745995

03/25/2023, 08:22:29: total loss for 164000 steps : 971.0631853938103

03/25/2023, 08:24:55: total loss for 165000 steps : 968.5628808736801

03/25/2023, 08:27:21: total loss for 166000 steps : 958.4325271844864

03/25/2023, 08:29:48: total loss for 167000 steps : 953.04018253088

03/25/2023, 08:32:15: total loss for 168000 steps : 946.4941085577011

03/25/2023, 08:34:41: total loss for 169000 steps : 936.3270008563995

03/25/2023, 08:37:09: total loss for 170000 steps : 930.9912779927254

03/25/2023, 08:39:35: total loss for 171000 steps : 923.7492192387581

03/25/2023, 08:42:02: total loss for 172000 steps : 919.7524968981743

03/25/2023, 08:44:28: total loss for 173000 steps : 909.9249004721642

03/25/2023, 08:46:55: total loss for 174000 steps : 903.509499669075

03/25/2023, 08:49:22: total loss for 175000 steps : 899.4685421586037

03/25/2023, 08:51:48: total loss for 176000 steps : 888.5117920041084

03/25/2023, 08:54:13: total loss for 177000 steps : 886.5494872331619

03/25/2023, 08:56:08: ---------- start evaluation after epoch 11 ----------

03/25/2023, 09:06:28: validation hit @ 1 is 0.011573683914826676

03/25/2023, 09:06:28: validation hit @ 5 is 0.029664587898196527

03/25/2023, 09:06:28: validation hit @ 10 is 0.04180010112927693

03/25/2023, 09:06:28: validation ncdg @ 5 is 0.0208807594236876

03/25/2023, 09:06:28: validation ncdg @ 10 is 0.0208807594236876

03/25/2023, 09:06:33: validation hit @ 1 is 0.00584302488903871

03/25/2023, 09:06:33: validation hit @ 5 is 0.01573122085510422

03/25/2023, 09:06:33: validation hit @ 10 is 0.023428282487780214

03/25/2023, 09:06:33: validation ncdg @ 5 is 0.010789948849357698

03/25/2023, 09:06:33: validation ncdg @ 10 is 0.010789948849357698

03/25/2023, 09:16:44: test hit @ 1 is 0.006573403000168549

03/25/2023, 09:16:44: test hit @ 5 is 0.016349233102983314

03/25/2023, 09:16:44: test hit @ 10 is 0.024608124051913028

03/25/2023, 09:16:44: test ncdg @ 5 is 0.011577288738556924

03/25/2023, 09:16:44: test ncdg @ 10 is 0.011577288738556924

03/25/2023, 09:16:51: test hit @ 1 is 0.004101354008652172

03/25/2023, 09:16:51: test hit @ 5 is 0.010618574077195348

03/25/2023, 09:16:51: test hit @ 10 is 0.01651778189785943

03/25/2023, 09:16:51: test ncdg @ 5 is 0.007363654264645672

03/25/2023, 09:16:51: test ncdg @ 10 is 0.007363654264645672

03/25/2023, 09:16:51: ---------- training epoch 12 ----------

03/25/2023, 09:17:34: total loss for 178000 steps : 878.7196144461632

03/25/2023, 09:20:00: total loss for 179000 steps : 872.6183027029037

03/25/2023, 09:22:25: total loss for 180000 steps : 858.7841667532921

03/25/2023, 09:24:50: total loss for 181000 steps : 855.9559269547462

03/25/2023, 09:27:15: total loss for 182000 steps : 850.9500839114189

03/25/2023, 09:29:39: total loss for 183000 steps : 844.4222279191017

03/25/2023, 09:32:04: total loss for 184000 steps : 838.2129778265953

03/25/2023, 09:34:28: total loss for 185000 steps : 831.1954792737961

03/25/2023, 09:36:53: total loss for 186000 steps : 823.1812939047813

03/25/2023, 09:39:18: total loss for 187000 steps : 819.6461032032967

03/25/2023, 09:41:43: total loss for 188000 steps : 813.3852053284645

03/25/2023, 09:44:08: total loss for 189000 steps : 804.838833630085

03/25/2023, 09:46:33: total loss for 190000 steps : 797.9262074828148

03/25/2023, 09:48:57: total loss for 191000 steps : 794.7584861516953

03/25/2023, 09:51:22: total loss for 192000 steps : 788.5722050666809

03/25/2023, 09:52:37: ---------- start evaluation after epoch 12 ----------

03/25/2023, 10:02:51: validation hit @ 1 is 0.010787122872071465

03/25/2023, 10:02:51: validation hit @ 5 is 0.026743075453677174

03/25/2023, 10:02:51: validation hit @ 10 is 0.03865385695825608

03/25/2023, 10:02:51: validation ncdg @ 5 is 0.019134892188354714

03/25/2023, 10:02:51: validation ncdg @ 10 is 0.019134892188354714

03/25/2023, 10:02:59: validation hit @ 1 is 0.00533737850441036

03/25/2023, 10:02:59: validation hit @ 5 is 0.014551379290971403

03/25/2023, 10:02:59: validation hit @ 10 is 0.020394404180010112

03/25/2023, 10:02:59: validation ncdg @ 5 is 0.010022336975525117

03/25/2023, 10:02:59: validation ncdg @ 10 is 0.010022336975525117

03/25/2023, 10:13:13: test hit @ 1 is 0.006011573683914827

03/25/2023, 10:13:13: test hit @ 5 is 0.014045732906343053

03/25/2023, 10:13:13: test hit @ 10 is 0.022248440923647396

03/25/2023, 10:13:13: test ncdg @ 5 is 0.010175569070505689

03/25/2023, 10:13:13: test ncdg @ 10 is 0.010175569070505689

03/25/2023, 10:13:31: test hit @ 1 is 0.0033709758975223325

03/25/2023, 10:13:31: test hit @ 5 is 0.009270183718186414

03/25/2023, 10:13:31: test hit @ 10 is 0.014439013427720658

03/25/2023, 10:13:31: test ncdg @ 5 is 0.0064187672448906535

03/25/2023, 10:13:31: test ncdg @ 10 is 0.0064187672448906535

03/25/2023, 10:13:31: ---------- training epoch 13 ----------

03/25/2023, 10:14:42: total loss for 193000 steps : 781.1397175788879

03/25/2023, 10:17:08: total loss for 194000 steps : 775.9450064301491

03/25/2023, 10:19:35: total loss for 195000 steps : 768.818577349186

03/25/2023, 10:22:01: total loss for 196000 steps : 764.0559263825417

03/25/2023, 10:24:25: total loss for 197000 steps : 759.9533911943436

03/25/2023, 10:26:49: total loss for 198000 steps : 750.7558131217957

03/25/2023, 10:29:13: total loss for 199000 steps : 748.2962327003479

03/25/2023, 10:31:35: total loss for 200000 steps : 742.9239305257797

03/25/2023, 10:33:58: total loss for 201000 steps : 734.5586375594139

03/25/2023, 10:36:20: total loss for 202000 steps : 731.6051931977272

03/25/2023, 10:38:43: total loss for 203000 steps : 724.6095151305199

03/25/2023, 10:41:06: total loss for 204000 steps : 722.4707497358322

03/25/2023, 10:43:30: total loss for 205000 steps : 715.267785191536

03/25/2023, 10:45:55: total loss for 206000 steps : 713.5253140926361

03/25/2023, 10:48:21: total loss for 207000 steps : 704.4989960193634

03/25/2023, 10:49:09: ---------- start evaluation after epoch 13 ----------

03/25/2023, 10:59:28: validation hit @ 1 is 0.010618574077195348

03/25/2023, 10:59:28: validation hit @ 5 is 0.027304904769930895

03/25/2023, 10:59:28: validation hit @ 10 is 0.03753019832574864

03/25/2023, 10:59:28: validation ncdg @ 5 is 0.019212938605757273

03/25/2023, 10:59:28: validation ncdg @ 10 is 0.019212938605757273

03/25/2023, 10:59:36: validation hit @ 1 is 0.004157536940277544

03/25/2023, 10:59:36: validation hit @ 5 is 0.012584976684083376

03/25/2023, 10:59:36: validation hit @ 10 is 0.01904601382100118

03/25/2023, 10:59:36: validation ncdg @ 5 is 0.008370225763095982

03/25/2023, 10:59:36: validation ncdg @ 10 is 0.008370225763095982

03/25/2023, 11:09:48: test hit @ 1 is 0.005955390752289454

03/25/2023, 11:09:48: test hit @ 5 is 0.014439013427720658

03/25/2023, 11:09:48: test hit @ 10 is 0.022416989718523513

03/25/2023, 11:09:48: test ncdg @ 5 is 0.010213043585891783

03/25/2023, 11:09:48: test ncdg @ 10 is 0.010213043585891783

03/25/2023, 11:10:04: test hit @ 1 is 0.0028091465812686107

03/25/2023, 11:10:04: test hit @ 5 is 0.008933086128434181

03/25/2023, 11:10:04: test hit @ 10 is 0.013540086521714702

03/25/2023, 11:10:04: test ncdg @ 5 is 0.005920778476334708

03/25/2023, 11:10:04: test ncdg @ 10 is 0.005920778476334708

03/25/2023, 11:10:04: ---------- training epoch 14 ----------

03/25/2023, 11:11:43: total loss for 208000 steps : 697.6824549436569

03/25/2023, 11:14:07: total loss for 209000 steps : 690.5571812987328

03/25/2023, 11:16:32: total loss for 210000 steps : 685.2501641213894

03/25/2023, 11:18:56: total loss for 211000 steps : 682.4913168549538

03/25/2023, 11:21:20: total loss for 212000 steps : 676.6518560647964

03/25/2023, 11:23:42: total loss for 213000 steps : 672.6292371153831

03/25/2023, 11:26:06: total loss for 214000 steps : 663.9969103038311

03/25/2023, 11:28:31: total loss for 215000 steps : 662.0554641187191

03/25/2023, 11:30:56: total loss for 216000 steps : 657.9121072888374

03/25/2023, 11:33:21: total loss for 217000 steps : 652.5546779036522

03/25/2023, 11:35:45: total loss for 218000 steps : 649.4349251389503

03/25/2023, 11:38:11: total loss for 219000 steps : 639.7581863105297

03/25/2023, 11:40:35: total loss for 220000 steps : 638.9178022146225

03/25/2023, 11:42:59: total loss for 221000 steps : 634.0486693382263

03/25/2023, 11:45:25: total loss for 222000 steps : 628.9775356352329

03/25/2023, 11:45:44: ---------- start evaluation after epoch 14 ----------

03/25/2023, 11:55:57: validation hit @ 1 is 0.009719647171189392

03/25/2023, 11:55:57: validation hit @ 5 is 0.024214843530535424

03/25/2023, 11:55:57: validation hit @ 10 is 0.03399067363335019

03/25/2023, 11:55:57: validation ncdg @ 5 is 0.01721968440006902

03/25/2023, 11:55:57: validation ncdg @ 10 is 0.01721968440006902

03/25/2023, 11:56:06: validation hit @ 1 is 0.0035957076240238214

03/25/2023, 11:56:06: validation hit @ 5 is 0.01168604977807742

03/25/2023, 11:56:06: validation hit @ 10 is 0.01702342828248778

03/25/2023, 11:56:06: validation ncdg @ 5 is 0.007685416403722782

03/25/2023, 11:56:06: validation ncdg @ 10 is 0.007685416403722782

03/25/2023, 12:06:07: test hit @ 1 is 0.004944097983032755

03/25/2023, 12:06:07: test hit @ 5 is 0.012304062025956514

03/25/2023, 12:06:07: test hit @ 10 is 0.020169672453508623

03/25/2023, 12:06:07: test ncdg @ 5 is 0.008580507855845384

03/25/2023, 12:06:07: test ncdg @ 10 is 0.008580507855845384

03/25/2023, 12:06:23: test hit @ 1 is 0.0025282319231417494

03/25/2023, 12:06:23: test hit @ 5 is 0.008483622675431204

03/25/2023, 12:06:23: test hit @ 10 is 0.01370863531659082

03/25/2023, 12:06:23: test ncdg @ 5 is 0.005555023915374866

03/25/2023, 12:06:23: test ncdg @ 10 is 0.005555023915374866

03/25/2023, 12:06:23: ---------- training epoch 15 ----------

03/25/2023, 12:08:28: total loss for 223000 steps : 622.7710821032524

03/25/2023, 12:10:51: total loss for 224000 steps : 620.9154339134693

03/25/2023, 12:13:14: total loss for 225000 steps : 616.7917878031731

03/25/2023, 12:15:39: total loss for 226000 steps : 608.4294527173042

03/25/2023, 12:18:02: total loss for 227000 steps : 608.0983311831951

03/25/2023, 12:20:22: total loss for 228000 steps : 601.6606380939484

03/25/2023, 12:22:42: total loss for 229000 steps : 596.7532267570496

03/25/2023, 12:25:03: total loss for 230000 steps : 595.0686884522438

03/25/2023, 12:27:23: total loss for 231000 steps : 591.6561165452003

03/25/2023, 12:29:44: total loss for 232000 steps : 587.4560499489307

03/25/2023, 12:32:04: total loss for 233000 steps : 583.5418926775455

03/25/2023, 12:34:25: total loss for 234000 steps : 575.1317911148071

03/25/2023, 12:36:46: total loss for 235000 steps : 573.9668118059635

03/25/2023, 12:39:06: total loss for 236000 steps : 570.1479779183865

03/25/2023, 12:41:18: ---------- start evaluation after epoch 15 ----------

03/25/2023, 12:51:15: validation hit @ 1 is 0.008034159222428226

03/25/2023, 12:51:15: validation hit @ 5 is 0.023821563009157817

03/25/2023, 12:51:15: validation hit @ 10 is 0.03404685656497556

03/25/2023, 12:51:15: validation ncdg @ 5 is 0.016040064495709553

03/25/2023, 12:51:15: validation ncdg @ 10 is 0.016040064495709553

03/25/2023, 12:51:23: validation hit @ 1 is 0.003146244171020844

03/25/2023, 12:51:23: validation hit @ 5 is 0.010843305803696838

03/25/2023, 12:51:23: validation hit @ 10 is 0.01646159896623406

03/25/2023, 12:51:23: validation ncdg @ 5 is 0.006991715362480381

03/25/2023, 12:51:23: validation ncdg @ 10 is 0.006991715362480381

03/25/2023, 13:01:18: test hit @ 1 is 0.005168829709534244

03/25/2023, 13:01:18: test hit @ 5 is 0.011910781504578909

03/25/2023, 13:01:18: test hit @ 10 is 0.02011348952188325

03/25/2023, 13:01:18: test ncdg @ 5 is 0.008481924571626815

03/25/2023, 13:01:18: test ncdg @ 10 is 0.008481924571626815

03/25/2023, 13:01:36: test hit @ 1 is 0.002359683128265633

03/25/2023, 13:01:36: test hit @ 5 is 0.008202708017304343

03/25/2023, 13:01:36: test hit @ 10 is 0.011124220461823698

03/25/2023, 13:01:36: test ncdg @ 5 is 0.00537286371143636

03/25/2023, 13:01:36: test ncdg @ 10 is 0.00537286371143636

03/25/2023, 13:01:36: ---------- training epoch 16 ----------

03/25/2023, 13:01:44: total loss for 237000 steps : 567.5080096721649

03/25/2023, 13:04:07: total loss for 238000 steps : 556.96839427948

03/25/2023, 13:06:30: total loss for 239000 steps : 557.2256788313389

03/25/2023, 13:08:54: total loss for 240000 steps : 551.9659865796566