Predictive Modeling-Based Non-Invasive System for Myeloproliferative Neoplasms Detection via Peripheral Blood Sampling

I am currently working on a research project focused on developing a Predictive Modeling-Based Non-Invasive System for Myeloproliferative Neoplasms Detection via Peripheral Blood Sampling. I have a CSV dataset containing 412 samples of healthy individuals and MPN patients, including gene expression data, sex, and age. I am seeking your expertise to assist with the following tasks: 1. Organizing and Harmonizing the Data: - Ensure all gene identifiers under the ID_ref column, such as (1007_s_at) and (TC04002953.hg.1), are consistent and harmonized across the dataset, and document each gene’s actual meaning before harmonizing them, I need to know what each new gene identifier reference to. - Properly label and organize the dataset to distinguish healthy samples from MPN patients using "healthy" for healthy individuals and the specific MPN subtype for diagnosed patients. 2. Feature Engineering to Identify Key Genes: - Identify the most important genes that indicate the existence of MPNs and the specific subtype using feature engineering techniques : Random Forest Feature Importance. - Provide a list of these genes along with their significance in MPN detection as a data graph. 3. Training a Machine Learning Model: - Train a Random Forest model using an 80/20 split to predict whether a person has an MPN. - If MPN is detected, classify the specific type of MPN. - Use gene expressions, sex, and age as features for the model. 4. Delivering Results: - The output should indicate: - Whether a person is infected with MPNs or not. - If infected, the specific type of MPN along with a likelihood percentage. - Provide an explanation of the results, including the key genes that contributed to the prediction and their importance in the model's decision-making process. Notes: - Missing Data Handling: Implement strategies to address any missing values in the dataset, such as imputation methods or exclusion criteria, to ensure data integrity. - Implement data visualizations for the following performance metrics: Confusion Matrix, Accuracy, Precision, Recall, F1 Score, ROC Curve with AUC, Specificity, Matthews Correlation Coefficient (MCC), Logarithmic Loss, and Calibration Curve. 25 samples : [login to view URL] 116 samples : [login to view URL] 181 samples : [**[login to view URL]**]([login to view URL]) 54 samples : [login to view URL] 36 samples : [login to view URL] total samples = 412 samples each sample file has specific AGE , SEX , SOURCE , TISSUE , DIAGNOSIS and GENOTYPE each single sample should have a CSV file with these columns and their information after organizing each sample with its diagnosis , age and sex etc.... all samples should be merged into a one data set like the screen shot I provided , and since data set 36 and 25 have different gene identifiers than those in data set 54 , 116 and 181 all gene id should be harmonized : 1. Obtain a Gene Annotation Source: Use a trusted gene database like HGNC, Ensembl, or NCBI Gene to obtain gene identifiers and their corresponding names. 2. Map Gene Identifiers: Use a tool or script (e.g., biomaRt in R, pybiomart in Python) to map harmonized gene IDs (e.g., Ensembl IDs) to gene symbols. 3. Verify Gene Names: Use the mapped gene symbols to cross-reference with the original source (HGNC/Ensembl) to ensure accuracy.