Skip to content

dli12/KDD20-On-Sampling-Top-K-Recommendation-Evaluation

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
figures
figures
 
 
README.md
README.md
 
 

Repository files navigation

On-sampling-Top-K-Recommendation-Evaluation

- FIGURE

1. (Figure 1)

2. (Figure 2 - 6)

3. (Figure 7)

4. (Figure 8)

- TABLE

1. Varing Sampling Top k (Table 3, Table 4)

  • Varing Sampling Top k on MultiVAE:ml-1m:n=100
n = 100, MultiVAE-ml-1m,binom n = 100, MultiVAE-ml-1m,hyper n = 100, MultiVAE-ml-1m,actual
  SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5
1 0.1031 0.0007 0.0442 0.1116 0.0829 1 0.1012 0.0007 0.0442 0.1116 0.0829 1 0.2101 0.0137 0.1608 0.2540 0.2161
2 0.2041 0.1116 0.1700 0.2185 0.2012 2 0.1964 0.1116 0.1700 0.2185 0.2012 2 0.3349 0.2540 0.3199 0.3810 0.3555
5 0.4157 0.3745 0.4050 0.4334 0.4209 5 0.4119 0.3745 0.4050 0.4334 0.4209 5 0.5533 0.5359 0.5603 0.5889 0.5768
10 0.6200 0.6048 0.6204 0.6323 0.6258 10 0.6157 0.6048 0.6204 0.6323 0.6258 10 0.7164 0.7192 0.7295 0.7391 0.7334
20 0.7992 0.7952 0.8012 0.8050 0.8028 20 0.7975 0.7952 0.8012 0.8050 0.8028 20 0.8531 0.8608 0.8661 0.8690 0.8674
50 0.9651 0.9671 0.9682 0.9684 0.9682 50 0.9661 0.9671 0.9682 0.9684 0.9682 50 0.9757 0.9795 0.9800 0.9803 0.9798
  • Varing Sampling Top k on MultiVAE:ml-1m:n=1000
n = 1000, MultiVAE-ml-1m,binom n = 1000, MultiVAE-ml-1m,hyper n = 1000, MultiVAE-ml-1m,actual
  SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5
1 0.1091 0.0957 0.0998 0.1116 0.1089 1 0.1096 0.0957 0.0998 0.1116 0.1089 1 0.2417 0.2377 0.2407 0.2540 0.2495
2 0.2126 0.2099 0.2124 0.2185 0.2174 2 0.2126 0.2099 0.2124 0.2185 0.2174 2 0.3682 0.3669 0.3704 0.3810 0.3770
5 0.4281 0.4286 0.4306 0.4334 0.4318 5 0.4290 0.4286 0.4306 0.4334 0.4318 5 0.5776 0.5848 0.5861 0.5889 0.5874
10 0.6293 0.6296 0.6303 0.6316 0.6316 10 0.6315 0.6296 0.6303 0.6316 0.6316 10 0.7286 0.7376 0.7381 0.7389 0.7389
20 0.8036 0.8040 0.8043 0.8050 0.8046 20 0.8060 0.8040 0.8043 0.8050 0.8046 20 0.8570 0.8679 0.8684 0.8690 0.8689
50 0.9672 0.9682 0.9682 0.9684 0.9684 50 0.9677 0.9682 0.9682 0.9684 0.9684 50 0.9747 0.9800 0.9800 0.9803 0.9803
  • Varing Sampling Top k on MultiVAE:yelp:n=100
n = 100, MultiVAE-yelp,binom n = 100, MultiVAE-yelp,hyper n = 100, MultiVAE-yelp,actual
  SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5
1 0.2574 0.0002 0.2006 0.3111 0.2691 1 0.2586 0.0002 0.2006 0.3111 0.2691 1 0.2757 0.0049 0.2154 0.3237 0.2837
2 0.4050 0.3129 0.3879 0.4450 0.4194 2 0.4031 0.3129 0.3879 0.4450 0.4194 2 0.4137 0.3257 0.3987 0.4550 0.4297
5 0.6242 0.6044 0.6308 0.6518 0.6417 5 0.6241 0.6044 0.6308 0.6518 0.6417 5 0.6297 0.6104 0.6355 0.6550 0.6455
10 0.7748 0.7698 0.7789 0.7853 0.7820 10 0.7759 0.7698 0.7789 0.7853 0.7820 10 0.7781 0.7722 0.7812 0.7876 0.7834
20 0.8913 0.8902 0.8933 0.8952 0.8941 20 0.8915 0.8902 0.8933 0.8952 0.8941 20 0.8917 0.8909 0.8941 0.8959 0.8950
50 0.9746 0.9752 0.9755 0.9755 0.9755 50 0.9749 0.9752 0.9755 0.9755 0.9755 50 0.9749 0.9754 0.9757 0.9757 0.9756
  • Varing Sampling Top k on MultiVAE:yelp:n=1000
n = 1000, MultiVAE-yelp,binom n = 1000, MultiVAE-yelp,hyper n = 1000, MultiVAE-yelp,actual
  SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5   SHR Linear Bound Beta@1 Beta@0.5
1 0.3049 0.2929 0.3005 0.3111 0.3070 1 0.3033 0.2929 0.3005 0.3111 0.3070 1 0.3155 0.3059 0.3144 0.3237 0.3193
2 0.4433 0.4349 0.4407 0.4450 0.4434 2 0.4413 0.4349 0.4407 0.4450 0.4434 2 0.4513 0.4458 0.4502 0.4550 0.4528
5 0.6488 0.6476 0.6494 0.6518 0.6507 5 0.6495 0.6476 0.6494 0.6518 0.6507 5 0.6519 0.6511 0.6533 0.6550 0.6543
10 0.7854 0.7838 0.7846 0.7853 0.7850 10 0.7851 0.7838 0.7846 0.7853 0.7850 10 0.7869 0.7862 0.7869 0.7876 0.7872
20 0.8952 0.8948 0.8950 0.8952 0.8951 20 0.8949 0.8948 0.8950 0.8952 0.8951 20 0.8960 0.8954 0.8956 0.8959 0.8957
50 0.9754 0.9755 0.9755 0.9755 0.9755 50 0.9756 0.9755 0.9755 0.9755 0.9755 50 0.9759 0.9756 0.9757 0.9757 0.9757

2. Mapping Functions Comparison (table 5)

MultiVAE:ml-1m

  MultiVAE:ml-1m
  abs rel abs@1 rel@1 abs@2-10 rel@2-10
Linear 0.0058 0.0226 0.0992 0.9934 0.0391 0.1238
Bound 0.0024 0.0103 0.0596 0.5970 0.0100 0.0363
Beta@1 0.0034 0.0074 0.0118 0.1177 0.0165 0.0440
Beta@0.5 0.0019 0.0043 0.0169 0.1692 0.0052 0.0112
Beta@0.2 0.0018 0.0065 0.0356 0.3566 0.0060 0.0207
Beta@P 0.0018 0.0052 0.0250 0.2504 0.0052 0.0146

NeuMF:yelp

  NeuMF:yelp
  abs rel abs@1 rel@1 abs@2-10 rel@2-10
Linear 0.0057 0.0164 0.2329 0.9886 0.0300 0.0632
Bound 0.0019 0.0046 0.0606 0.2571 0.0061 0.0125
Beta@1 0.0038 0.0070 0.0414 0.1756 0.0221 0.0413
Beta@0.5 0.0020 0.0030 0.0032 0.0134 0.0112 0.0197
Beta@0.2 0.0015 0.0027 0.0253 0.1075 0.0057 0.0097
Beta@P 0.0015 0.0024 0.0145 0.0617 0.0067 0.0106
  • EASE:ml-1m
  EASE:ml-1m
  abs rel abs@1 rel@1 abs@2-10 rel@2-10
Linear 0.0056 0.0211 0.1101 0.9666 0.0374 0.1118
Bound 0.0025 0.0084 0.0500 0.4390 0.0092 0.0294
Beta@1 0.0037 0.0081 0.0205 0.1802 0.0152 0.0408
Beta@0.5 0.0019 0.0031 0.0055 0.0480 0.0040 0.0085
Beta@0.2 0.0018 0.0047 0.0237 0.2078 0.0047 0.0137
Beta@P 0.0017 0.0034 0.0141 0.1235 0.0027 0.0059
  • itemKNN-yelp
  itemKNN:yelp
  abs rel abs@1 rel@1 abs@2-10 rel@2-10
Linear 0.0057 0.0145 0.3019 0.9874 0.0240 0.0443
Bound 0.0019 0.0035 0.0529 0.1731 0.0053 0.0086
Beta@1 0.0040 0.0069 0.0612 0.2003 0.0230 0.0384
Beta@0.5 0.0024 0.0036 0.0183 0.0599 0.0131 0.0212
Beta@0.2 0.0015 0.0022 0.0113 0.0369 0.0067 0.0101
Beta@P 0.0017 0.0023 0.0023 0.0075 0.0086 0.0135

3. Predicting Winners(and Relative Performance) (table 6)

  • ml-1m
ml-1m NeuMF MultiVAE EASE ALS
k SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5
1 0.2081 0.1502 0.2058 0.2119 0.1608 0.2161 0.2233 0.1737 0.2326 0.2136 0.1606 0.2242
2 0.3268 0.3119 0.3470 0.3430 0.3199 0.3555 0.3497 0.3348 0.3700 0.3329 0.3205 0.3508
5 0.5485 0.5555 0.5669 0.5558 0.5603 0.5768 0.5646 0.5732 0.5873 0.5339 0.5452 0.5566
10 0.7156 0.7263 0.7313 0.7177 0.7295 0.7334 0.7207 0.7339 0.7374 0.6942 0.7053 0.7093
20 0.8508 0.8639 0.8646 0.8545 0.8661 0.8674 0.8477 0.8594 0.8604 0.8225 0.8364 0.8374
50 0.9722 0.9790 0.9790 0.9725 0.9800 0.9798 0.9555 0.9618 0.9616 0.9315 0.9364 0.9364
  • yelp
yelp NeuMF MultiVAE EASE ALS
k SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5
1 0.2346 0.1846 0.2488 0.2623 0.2154 0.2837 0.2751 0.2284 0.2953 0.2467 0.1990 0.2597
2 0.3697 0.3600 0.3921 0.4049 0.3987 0.4297 0.4181 0.4105 0.4423 0.3782 0.3698 0.3968
5 0.5934 0.6009 0.6114 0.6263 0.6355 0.6455 0.6328 0.6426 0.6538 0.5734 0.5826 0.5917
10 0.7537 0.7606 0.7640 0.7751 0.7812 0.7834 0.7770 0.7842 0.7878 0.7025 0.7049 0.7081
20 0.8814 0.8854 0.8862 0.8922 0.8941 0.8950 0.8867 0.8889 0.8894 0.7972 0.7994 0.7999
50 0.9756 0.9762 0.9762 0.9745 0.9757 0.9756 0.9574 0.9578 0.9577 0.8756 0.8759 0.8758
  • pinterest-20
pinterest-20 NeuMF MultiVAE EASE ALS
k SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5
1 0.2734 0.2019 0.2780 0.3106 0.2417 0.3219 0.2899 0.2223 0.2981 0.2623 0.1972 0.2651
2 0.4360 0.4095 0.4489 0.4792 0.4560 0.4950 0.4529 0.4275 0.4656 0.4158 0.3903 0.4269
5 0.7010 0.7080 0.7221 0.7296 0.7351 0.7477 0.7057 0.7120 0.7248 0.6659 0.6706 0.6833
10 0.8741 0.8832 0.8866 0.8876 0.8948 0.8977 0.8688 0.8776 0.8807 0.8298 0.8381 0.8416
20 0.9651 0.9685 0.9688 0.9706 0.9737 0.9739 0.9596 0.9633 0.9637 0.9233 0.9269 0.9272
50 0.9935 0.9934 0.9934 0.9946 0.9947 0.9947 0.9891 0.9894 0.9893 0.9582 0.9584 0.9583
  • citeulike
citeulike NeuMF MultiVAE EASE ALS
K SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5 SHR Bound Beta@0.5
1 0.3994 0.4277 0.5059 0.4886 0.5545 0.6157 0.4648 0.5118 0.5779 0.2801 0.2508 0.3122
2 0.5837 0.6069 0.6325 0.6756 0.7074 0.7271 0.6411 0.6745 0.6970 0.4208 0.4174 0.4460
5 0.7674 0.7748 0.7838 0.8285 0.8398 0.8458 0.8024 0.8117 0.8150 0.6193 0.6269 0.6374
10 0.8717 0.8780 0.8800 0.9108 0.9142 0.9161 0.8804 0.8860 0.8876 0.7620 0.7678 0.7696
20 0.9407 0.9442 0.9447 0.9613 0.9629 0.9631 0.9342 0.9360 0.9362 0.8694 0.8707 0.8714
50 0.9870 0.9876 0.9876 0.9924 0.9919 0.9919 0.9690 0.9688 0.9688 0.9600 0.9584 0.9580

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published