Skip to content

High performance rank executor for advertisement and recommendation system, implemented in C/C++ and support ensembled into Java/Scala host as plguin or independent process, and support pytorch deep learning model.

License

Notifications You must be signed in to change notification settings

niuchenglei/rankextor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

High performance rank executor for advertisement and recommendation system, implemented in C/C++ and support ensembled into Java/Scala host as plguin or independent process, and it support ML models trained by Xgboost in Python or event Tensorflow deep learning models.

Rankextor is a set of algorithm execution engines based on C/C++. It is mainly used for algorithm serving in the fields of recommendation, advertising, and search. It supports embedding into Java/Scala service processes or providing RPC ranking services as an independent process.

Features

  1. Flexible and efficient "graph" execution engine.

  2. Convenient feature, strategy, model, and feature engineering support.

    • Features such as:
      • Basic user portrait characteristics
      • Statistical feedback CTR features
      • Material side characteristics
    • Click-through rate estimation algorithm is as follows:
      • LR
      • FM
      • GBDT+LR
      • Deep Learning (Pytorch)
    • Feature engineering processing such as:
      • Equal frequency, equal distance scattering
      • Bucket
      • Feature crossover
      • Multiple feature support
      • Sample sampling
      • hash discrete
    • Strategies such as:
      • Scoring and ranking
      • Sort and adjust
      • Material filtration
  3. Embed Java services, or provide RPC services as independent processes

  4. Dynamic loading of model files and configuration files

installation steps

cmake ..
make

For pytorch model training and serving part, check the pytorch_ranking

DAG Configuration

The following configuration defines what kind of features we need, and what models shall take charge and what kind of rank strategies are needed.

name: "model"
condition {
    name:"bidfeed"
}
condition {
    name:"cpm"
}
condition {
    name:"cpc"
}
condition {
    name:"ocpm"
}
feature {
    name: "request_feature"
    data: { key: "redis_user_schema" value: "redis_user_schema.txt" }
    group: "begin"
}
extractor {
    name: "xfea1"
    type: "xfea"
    bottom: "begin"
    top: "ctcvr_fea"
    config: "suning_features.conf"
}
model {
    name: "gds_ctcvr1"
    type: "fm"
    bottom: "ctcvr_fea"
    top: "ctcvr"
    role: PREDICTOR
    model_file: "fm.model"
} 
strategy {
    name: "reset"
    order: -20
}
strategy {
    name: "calibrate"
    order: 1
    ids: "1"
    desc: "cut"
}
strategy {
    name: "ocpx"
    order: 2
}
strategy {
    name: "rankscore"
    order: 3
}
strategy {
    name: "sort"
    order: 4
}

About

High performance rank executor for advertisement and recommendation system, implemented in C/C++ and support ensembled into Java/Scala host as plguin or independent process, and support pytorch deep learning model.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published