Neural Network Programming - Part 3

Overview

Languages used: PHP 5.6, Python 3

Today, I want to talk about porting the neural network over to Python. Some of the mistakes I made, lessons I learned, and most importantly a performance comparison. In case you're curious, I found Python not only to move at a snail’s pace compared to PHP, but to perform so horrendously I couldn't even finish my benchmarks. Did I do something wrong? Is my code un-optimized? Read on and find out!

The Porting Process

Ah, porting code from one language to another language has got to be one of the most tedious, brain-dead tasks in existence. Needless to say, I popped on some tunes and got to work. There's nothing quite like writing Python code to the background track of Brittney Spears.

The structure of the Python code was nearly identical to that of the PHP code. The only changes were with respect to initializing Python lists and the syntax of the code. You can find the codes at the bottom of this article if you're interested.

Benchmarking

To put it simply, Python couldn't finish 1 000 000 * 29 iterations in the same time it took PHP to complete 15 000 000 * 29 iterations. In fact, as I'm writing this, Python still hasn't reached 1 000 000 * 29 iterations. So, is Python really this slow, is there a problem with the code or is there a fundamental difference between the two languages which causes such a huge discrepancy? After all, my hypothesis coming in to this was that Python should've been roughly 4-5 times faster than PHP.

Finding Sources of Error

After doing quick research online:

Language CPU time CPU time CPU time Slower than Slower than Language Version
User System Total C++ Previous
C++ (optimized with -O2) 0.952 0.172 1.124 g++ 5.3.1
Java 8 (non-std lib) 1.332 0.096 1.428 27% 27% 1.8.0_72
Python 2.7 + PyPy 1.560 0.160 1.720 53% 20% PyPy 4.0.1
Javascript (nodejs) 1.524 0.516 2.040 81% 19% 4.2.6
C++ (not optimized) 2.988 0.168 3.156 181% 55% g++ 5.3.1
PHP 7.0 6.524 0.184 6.708 497% 113% 7.0.2
Java 8 14.616 0.908 15.524 1281% 131% 1.8.0_72
Python 3.5 18.656 0.348 19.004 1591% 22% 3.5.1
Python 2.7 20.776 0.336 21.112 1778% 11% 2.7.11
Perl 25.044 0.236 25.280 2149% 20% 5.22.1
PHP 5.6 66.444 2.340 68.784 6020% 172% 5.6.17

Source: Famzah.net

And a few other various articles confirmed my suspicions: Stack Overflow, Quora

Something must be wrong! And, as it turns out, something was drastically wrong in the PHP script. It wasn't Python which was running too slow, but PHP was running far too quickly!

The Source of the Error

Well, I ran some quick debug operations on the PHP script, in particular outputting the amount of iterations it was doing. When the PHP script finished, it output it ran of a total of 435 500 iterations. This didn't make sense at all! It should be running for around 15 000 000 * 29 = 435 000 000 iterations. Strangely it was exactly a factor of 1000 off of where it should've been.

After looking through the code I noticed a problem at the start of the train function. See if you can find my mistake:

1cbe62e1c06d1a5bfb094fd714bfd86b.png

When I was removing the debug text, I simply commented out the echo statements in the PHP code. However, this introduced a flaw which not only broke the training of the network, but also explained some inconsistent outputs I couldn't explain during testing. The if statement I was using to output which 1000th iteration we were on was controlling the for loop afterwards because I had commented out the line of code it used to use.

This could have all been avoided if I had taken 1 of 2 actions:

  • If I had simply removed control statements for the debug text as well.
  • If I had used {} braces for all of my control statements rather than those with only one line of code underneath them.

After remedying the problem, I reran the PHP script and it seems to be taking much longer to run now. In fact, long enough that neither the Python nor PHP scripts have finished yet.

However, a preliminary test on the PHP script showed that after it ran 10000 iterations properly, the margin of error was up to 1 000 000 times smaller. This is a good indication of the PHP script and the Python script acting similarly as the Python script output a similar source of error after 10000 iterations.

In case you were curious, here is how I remedied the error:

0f1ad74a7c300f38fb7f279187fca02f.png

PHP Neural Network, Python Neural Network

Thanks for reading!


Shawn Clake

Freelance Developer

Software Engineering Student - U of R

Current: Assistant to Manager of Instructional Tech - U of R

Web|Unreal|C++|Java|Python|Go

Web: http://shawnclake.com

Email: shawn.lavawater@gmail.com

Email 2: shawn.clake@gmail.com

Posted in Technology on Feb 14, 2017