feat: Add Neural Network (Multilayer Perceptron) #1025

imdeep2905 · 2020-08-14T19:30:50Z

Description of Change

Added Neural Network as machine_learning/neural_network.cpp
Added Vector Ops as machine_learning/vector_ops.hpp (Operations required for Neural Network)
Added iris.csv as machine_learning/iris.csv (Training file for Network)

This will close #967

Checklist

Added description of change
Added file name matches File name guidelines
Added tests and example, test must pass
Added documentation so that the program is self-explanatory and educational - Doxygen guidelines
Relevant documentation/comments is changed or added
PR title follows semantic commit guidelines
Search previous suggestions before making a new one, as yours may be a duplicate.
I acknowledge that all my contributions will be made under the project's license.

Notes:

Updating fork

Updating Fork

Updating fork

machine_learning/iris.csv

imdeep2905 · 2020-08-14T19:45:02Z

Is there any way to solve following :

/home/runner/work/C-Plus-Plus/C-Plus-Plus/machine_learning/vector_ops.hpp:163:10: error: no template named 'uniform_real_distribution' in namespace 'std'; did you mean 'uniform_int_distribution'? [clang-diagnostic-error]
    std::uniform_real_distribution <T> distribution(low, high);

machine_learning/vector_ops.hpp

machine_learning/neural_network.cpp

machine_learning/vector_ops.hpp

Co-authored-by: David Leal <halfpacho@gmail.com>

machine_learning/neural_network.cpp

machine_learning/vector_ops.hpp

kvedala

👍 Excellent work Thanks 😄
Please note the comments.
Reviewed file neural_network.cpp. Haven't reviewed vector_ops.hpp and haven't executed the code yet.

machine_learning/iris.csv

machine_learning/neural_network.cpp

kvedala · 2020-08-15T15:27:56Z

machine_learning/neural_network.cpp

+                 * @param file_name file from which model will be loaded (*.model)
+                 * @return instance of NeuralNetwork class with pretrained weights
+                 */
+                NeuralNetwork load_model (const std::string &file_name) {


Since this function returns a class instance, for it to be usable, it'd require the implementation of a copy constructor for this class.

I have tested it with default copy constructor and it works fine.

kvedala

Sorry for the delayed response. I was waiting to get a change to execute and test the code. I did on GitPod using the link in the description and got the result below. I think there is some error in the code. Can you please review and fix. Thanks.

gitpod /workspace/C-Plus-Plus $ /workspace/C-Plus-Plus/build/machine_learning/neural_network
INFO: Network constructed successfully
===============================================================
                + MODEL SUMMARY +
===============================================================
1) Neurons : 4, Activation : none, Kernal Shape : (4, 4)
2) Neurons : 6, Activation : relu, Kernal Shape : (4, 6)
3) Neurons : 3, Activation : sigmoid, Kernal Shape : (6, 3)
===============================================================
INFO: Training Started
Training: Epoch 1/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 2/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 3/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 4/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 5/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 6/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 7/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 8/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 9/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 10/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 11/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 12/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 13/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 14/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 15/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 16/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 17/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 18/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 19/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 20/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 21/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 22/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 23/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 24/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 25/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 26/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 27/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 28/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 29/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 30/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 31/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 32/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 33/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 34/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 35/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 36/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 37/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 38/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 39/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 40/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 41/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 42/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 43/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 44/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 45/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 46/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 47/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 48/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 49/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 50/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 51/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 52/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 53/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 54/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 55/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 56/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 57/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 58/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 59/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 60/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 61/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 62/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 63/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 64/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 65/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 66/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 67/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 68/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 69/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 70/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 71/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 72/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 73/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 74/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 75/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 76/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 77/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 78/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 79/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 80/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 81/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 82/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 83/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 84/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 85/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 86/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 87/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 88/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 89/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 90/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 91/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 92/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 93/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 94/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 95/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 96/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 97/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 98/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 99/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
Training: Epoch 100/100, Loss: -nan, Accuracy: -nan, Taken time: 0 seconds
neural_network: ../machine_learning/neural_network.cpp:780: void test(): Assertion `machine_learning::argmax(myNN.single_predict({{6.4,2.9,4.3,1.3}})) == 1' failed.
Aborted (core dumped)
gitpod /workspace/C-Plus-Plus $

imdeep2905 · 2020-08-19T04:52:46Z

Sorry for the delayed response. I was waiting to get a change to execute and test the code. I did on GitPod using the link in the description and got the result below. I think there is some error in the code. Can you please review and fix. Thanks.

I did a quick run on gitpod and it works fine:

Panquesito7

I got the same output result as @kvedala's #1025 (review).

kvedala · 2020-08-19T18:44:16Z

Sorry for the delayed response. I was waiting to get a change to execute and test the code. I did on GitPod using the link in the description and got the result below. I think there is some error in the code. Can you please review and fix. Thanks.

I did a quick run on gitpod and it works fine:

Not sure how you ran the code. I tried again with all available compilers with the same error. Steps to obtain the error:

click on the open in gitpod link in description above
after the environment loads, it gives the option to pick the compiler toolkit. I selected "GCC 9.3.0" once, another time I selected "Clang 11.0.0".
Once the toolkit is selected, the environment executes configuration step on its own. If not, ignore and proceed to next step 4.
Click on "Select the target to launch" button.
This will open up a drop-down list of available targets. Type "neural" and the file will be displayed. Click and select it.
Compile and Run the target by clicking on the "Run" icon.
The program will compile and execute. Please report the results without any modification to the code.

imdeep2905 · 2020-08-19T19:00:23Z

The program will compile and execute. Please report the results without any modification to the code.

I have done as you said. I am actually getting some weird output. But when I run program with command line (i.e. g++ -std=c++14) I get output without any issues. I think the issue is with iris.csv. Probably program can't find iris.csv. Should I check again with absolute path rather than relative (for iris.csv) ?

imdeep2905 · 2020-08-19T19:11:51Z

Probably program can't find iris.csv. Should I check again with absolute path rather than relative (for iris.csv) ?

Yup issue indeed was with path to iris.csv. Now it is working fine.

Now, should I change path in this PR ?

kvedala · 2020-08-19T19:22:20Z

Probably program can't find iris.csv. Should I check again with absolute path rather than relative (for iris.csv) ?

Yup issue indeed was with path to iris.csv. Now it is working fine.

Now, should I change path in this PR ?

👍 Ah. no need to change the path. That is why I suggested to generate pattern data :)
Well done

imdeep2905 · 2020-08-19T19:27:41Z

Thank you very much !!! 🍾

One last thing. Actually I am bit new with this PR stuff. I can see review option on other PRs. Is it possible for me to do a review of PRs ?

kvedala · 2020-08-19T19:30:23Z

Thank you very much !!! 🍾

One last thing. Actually I am bit new with this PR stuff. I can see review option on other PRs. Is it possible for me to do a review of PRs ?

Absolutely. Please do provide your feedback while respecting other contributors :)
Thank you 🍻

kvedala · 2020-08-19T19:42:07Z

@imdeep2905 Can you create a PR with the following fix:

check if the file was opened successfully, if not, print error message to std::cerr and exit

imdeep2905 · 2020-08-19T19:43:53Z

@imdeep2905 Can you create a PR with the following fix:

check if the file was opened successfully, if not, print error message to std::cerr and exit

Sure !
On it's way...

kvedala · 2020-08-19T19:45:14Z

machine_learning/neural_network.cpp

+                                     const bool &normalize,
+                                     const int &slip_lines = 1) {
+                    std::ifstream in_file; // Ifstream to read file
+                    in_file.open(file_name.c_str(), std::ios::in); // Open file


here, you can add a check, something like this:

if(!in_file.is_open()) { std::cerr << "Unable to open file: "<< file_name << "\n"; exit(EXIT_FAILURE); }

imdeep2905 added 6 commits July 19, 2020 14:00

Merge pull request #1 from TheAlgorithms/master

6d9e3f9

Updating fork

Merge pull request #2 from TheAlgorithms/master

0e25c17

Updating Fork

Merge pull request #3 from TheAlgorithms/master

81260ae

Updating Fork

Merge pull request #4 from TheAlgorithms/master

26a78ce

Updating Fork

Merge pull request #5 from TheAlgorithms/master

6756d04

Updating fork

Completed NN

b80a14b

Panquesito7 added the enhancement New feature or request label Aug 14, 2020

Panquesito7 reviewed Aug 14, 2020

View reviewed changes

machine_learning/iris.csv Outdated Show resolved Hide resolved

Made changes

93a37ff

Panquesito7 requested changes Aug 14, 2020

View reviewed changes

machine_learning/vector_ops.hpp Show resolved Hide resolved

imdeep2905 added 2 commits August 15, 2020 01:27

Added return in identity function

938a20f

Added <random> and fixed namespace naming

fdc5597

Panquesito7 added the automated tests are failing Do not merge until tests pass label Aug 14, 2020

clang-tidy changes

4d3648a

imdeep2905 requested a review from Panquesito7 August 14, 2020 20:22

Panquesito7 removed the automated tests are failing Do not merge until tests pass label Aug 14, 2020

Panquesito7 requested changes Aug 14, 2020

View reviewed changes

Panquesito7 added the requested changes changes have been requested label Aug 14, 2020

imdeep2905 and others added 7 commits August 15, 2020 02:18

Update machine_learning/neural_network.cpp

f61d6c8

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/neural_network.cpp

cbf3430

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/neural_network.cpp

cebe0b1

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/vector_ops.hpp

fa02e0a

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/vector_ops.hpp

bafb999

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/neural_network.cpp

11cc006

Co-authored-by: David Leal <halfpacho@gmail.com>

Update machine_learning/neural_network.cpp

7217cb1

Co-authored-by: David Leal <halfpacho@gmail.com>

Panquesito7 requested changes Aug 14, 2020

View reviewed changes

machine_learning/neural_network.cpp Outdated Show resolved Hide resolved

Panquesito7 added automated tests are failing Do not merge until tests pass awaiting modification Do not merge until modifications are made labels Aug 14, 2020

kvedala reviewed Aug 14, 2020

View reviewed changes

machine_learning/vector_ops.hpp Outdated Show resolved Hide resolved

imdeep2905 requested review from Panquesito7 and kvedala August 15, 2020 05:56

kvedala reviewed Aug 15, 2020

View reviewed changes

imdeep2905 added 2 commits August 15, 2020 23:30

Done suggested changes

9e57e94

Fixed a comment

f632611

This comment has been minimized.

Sign in to view

clang-tidy fixes

d078dba

Panquesito7 removed automated tests are failing Do not merge until tests pass awaiting modification Do not merge until modifications are made labels Aug 16, 2020

imdeep2905 requested a review from kvedala August 18, 2020 18:06

kvedala requested changes Aug 18, 2020

View reviewed changes

imdeep2905 requested a review from kvedala August 19, 2020 04:53

Panquesito7 requested changes Aug 19, 2020

View reviewed changes

imdeep2905 requested a review from Panquesito7 August 19, 2020 19:15

kvedala approved these changes Aug 19, 2020

View reviewed changes

kvedala merged commit 4a34bec into TheAlgorithms:master Aug 19, 2020

Panquesito7 added approved Approved; waiting for merge and removed requested changes changes have been requested labels Aug 19, 2020

imdeep2905 deleted the nn branch August 19, 2020 19:38

kvedala reviewed Aug 19, 2020

View reviewed changes

imdeep2905 mentioned this pull request Aug 24, 2020

fix: error handling in opening of external files (neural_network.cpp). #1044

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add Neural Network (Multilayer Perceptron) #1025

feat: Add Neural Network (Multilayer Perceptron) #1025

imdeep2905 commented Aug 14, 2020 •

edited

Loading

imdeep2905 commented Aug 14, 2020

kvedala left a comment

kvedala Aug 15, 2020

imdeep2905 Aug 15, 2020

This comment has been minimized.

kvedala left a comment

imdeep2905 commented Aug 19, 2020

Panquesito7 left a comment •

edited

Loading

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020 •

edited

Loading

imdeep2905 commented Aug 19, 2020

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020

kvedala commented Aug 19, 2020

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020

kvedala Aug 19, 2020

feat: Add Neural Network (Multilayer Perceptron) #1025

feat: Add Neural Network (Multilayer Perceptron) #1025

Conversation

imdeep2905 commented Aug 14, 2020 • edited Loading

Description of Change

Checklist

imdeep2905 commented Aug 14, 2020

kvedala left a comment

Choose a reason for hiding this comment

kvedala Aug 15, 2020

Choose a reason for hiding this comment

imdeep2905 Aug 15, 2020

Choose a reason for hiding this comment

This comment has been minimized.

kvedala left a comment

Choose a reason for hiding this comment

imdeep2905 commented Aug 19, 2020

Panquesito7 left a comment • edited Loading

Choose a reason for hiding this comment

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020 • edited Loading

imdeep2905 commented Aug 19, 2020

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020

kvedala commented Aug 19, 2020

kvedala commented Aug 19, 2020

imdeep2905 commented Aug 19, 2020

kvedala Aug 19, 2020

Choose a reason for hiding this comment

imdeep2905 commented Aug 14, 2020 •

edited

Loading

Panquesito7 left a comment •

edited

Loading

imdeep2905 commented Aug 19, 2020 •

edited

Loading