In Which a Hardware Platform was Chosen, Part 2

How Much Computer Do You Need?

I buy computers based on best price per pound.

This is part two of two posts about how I decided on what kind of computing platform to use for my deep learning studies. If you haven't read it yet, the previous blog was In Which a Hardware Platform was Chosen, Part 1 (link), where I talked about the decision between cloud and personal device, and between desktop and laptop computers.

Laptops comes in many shapes and sizes nowadays, with different trade-offs in each design. Due to the GPU requirement, tiny laptops such as ultrabooks are out of the picture, as they do not have the space or the heat dissipation capacity of larger laptops. As this laptop is going to be my desktop replacement and my main computer, I wanted as much of a behemeth of a machine as possible, with the biggest screen and the highest specs available for a CPU and GPU that is within a reasonable price point. The largest laptops in production have 17" screens, which is a size that can accept for both desktop space, gaming, and video editing.

Which GPU?

This is not a GPU, but GRU was rendered using a GPU

In order to use CUDA acceleration, which many deep learning frameworks have enabled and optimized for, an Nvidia GPU is needed since CUDA is Nvidia proprietary technology. AMD Radeon GPUs support OpenCL, which although my preference is for open collaborative projects such as OpenCL (on principle, I use open source software such as use Gimp, Inkscape, and Audacity), performance overrides those concerns, and I shall stick with Nvidia GPUs.

Although actual performance is hard to judge, since each laptop's power budget and cooling efficiency is different, relative performance can be compared, holding everything else constant.

Numbers from Nvidia (GeForce link)

Class is a name I chose to signify what market segment each is targeting. CUDA cores refers to the number of physical unit of separate computation blocks each GPU has, the more cores there are, the more computation the GPU can run in parallel, which reduces the time needed to finish the computation. Memory refers to the dedicated graphics memory available to the GPU. This is important because of several reasons. First, it is physically located close to the GPU, so the speed can be higher. Second, it is completely reserved for the GPU so that the GPU does not need to compete with the CPU for space. Thus the more memory and higher bandwidth the better. For many deep learning applications, the training data size is quite large, so having a large graphics memory is very advantageous. Finally, the graphics performance is a number provided by Nvidia to give a rough general view of relative performance between the GPUs.

In terms of price-performance-ratio, the GTX 1060 is at the peak. However for a little more cash, the significant performance improvement of the GTX 1070 is worth it. The jump in price to the GTX 1080 was quite significant, as it always is for the top of the line. Therefore I decided to find a laptop with a GTX 1070.

Which CPU?

Bletchley Park's Colossus Mark 2 in Legos by James Pegrum (Flickr link)

As I have mention previously, CPUs are less important than GPUs for deep learning. However, it still figures into the picture, as not every part of neural network training is done on the GPU. Also, for other use cases, a CPU of sufficient performance is still important. Here I am once again sticking to the market-leading vendor, this time it is Intel, as there are more devices available, making comparison shopping easier.

The number of cores, similar to the GPU, indicate separate physical CPUs which can run in parallel. Threads can be thought of as virtual CPUs, where each thread can run in parallel with another, so that for CPUs which are not fully utilized by one thread, the remaining resources can be utilized by another thread. Finally, cache is the amount of fast memory physically attached to the CPU, which stores often-used data so that it can be retrieved quickly, as opposed to retrieving it from the slower main memory, or the very slow permanent storage such as hard drives or SSD drives.

The specs above may be true around the time this blog post is published, but Intel is known to introduce new variants of each line with different specs, some of which are more powerful than an older variant of a higher class, so please check the specs in more detail instead of just relying on the i3/i5/i7/i9 classification alone to make your decision. This is also why there is no straight-up performance comparison numbers for the CPUs. Not only is it dependent on power budget and cooling efficiency, but due to the overlap in performance between the classes, there is no one-number-fits-all for CPUs, even if all other variables are held constant.

As previously in the GPU section, best price-to-performance is around the mass market i5, but for a little extra money, the jump in performance to an i7 is worth it, while the jump to i9 is exorbitant, therefore I will look around for a recent version of an i7.

Other Components

Assortment of Clock Cogs and Gears by www.freeimages.co.uk (freeimages link)

I won't go into much detail, but some other items to consider are:

Main memory: The larger the memory, the more training data / multi-tasking you will be able to do without running out of memory or swapping to disk. I would suggest 16GB as a minimum. Memory is typically easily upgradable, so even if the machine does not have the maximum amount it can support already installed, you can always install it later, especially as prices comes down and make it worth the upgrade.
Storage: Solid state disks (SSD) are preferable for speed, but they are still much more expensive than traditional hard drives. Therefore I would suggest a SSD for the files you access often (the OS, applications, and any data files that will be used often), and a large hard disk for the long term storage of infrequently accessed files.

Which Laptop?

After deciding in general what kind of specs you want for the laptop, it's time to dive into comparing the actual devices from different manufacturers. There are just too many devices for me to delve into here in this blog, but in general a well-recognized brand which provide a well-built machine and easily-accessible software and drivers is preferable to me to reduce maintenance headaches, even for cases where the manufacturer provides no support, as a commonly available device will have users posting discussions of their problems and solutions online.

Accessories?

As long as you have a towel, any other accessories can be borrowed from strags (Fandom link)

With the amount of heat that the laptop will generate when doing processing intensive tasks, some sort of additional cooling would be helpful to keep the temperature down so that the processors doesn't go into thermal throttling, which will keep performance high while also prolonging the lifetime of the electronics.

Not knowing which accessories have the highest benefits, I went on eBay and picked out some items which may help, and then tested these items to see how much effect they had on the temperature.

Laptop stand with fans and adjustable angle

I purchased a laptop stand with multiple fans, which not only lifts the bottom of the laptop of the desk so that it can get better airflow, but also help push fresh air through the vents on the bottom of the laptop.

Many models similar to this vacuum fan can be found online

I also purchased two vacuum fans which can be pushed against the exhaust vents on the back of the laptop and help suck out the heated air at a fast rate than relying on the internal fans alone.

The process I ran the test was as follows:

Check ambient temperature
Let the laptop cool to a steady-state resting temperature while running some minimal web browsing
Run a processing-intensive task
Let the laptop warm up to a steady-state temperature while the task is running
place laptop on stand and turn on stand fans
Let the laptop cool down to a steady-state temperature while the task continues to run
Place the two vacuum fans at the vents on the back of the laptop
Let the laptop cool down to a steady-state temperature while the task continues to run

This process allows the separation of each state, and make the contribution of each accessory clear. I used the app NZXT CAM (NZXT link) to read the internal temperature sensors of the CPU and GPU, because it can show both and provide a temperature chart so that I can confirm visually that it has reached a steady-state temperature.

For the first run, the task I used was the training of a small neural network. This task was not a very high workload, but it still showed the effects of the accessories.

It appears that for this workload, the stand provides quite a bit of cooling for the GPU (4 degrees of cooling / 22 degrees of heat gain from rest = 18% versus no accessories) while the vacuum fan provides only a little (1 degrees of cooling / 22 degrees of heat gain from rest = 4.5%). But for ~$30, this kind of performance is quite acceptable.

For the second run, I installed the GPU benchmark Basemark GPU 1.1 (Basemark link), which generates game-level computer graphics using the latest techniques, and pegs the GPU load to 100%.

This workload stresses the CPU more than the previous one, and the combined cooling performance was significant (7/29 = 24%), while the GPU cooling was similar to before. Two interesting points:

It's clear that with a higher temperature, cooling has more of an effect, as can be seen by the CPU being cooled down to similar levels as in the first run (the lower final temperature may be due to lower ambient temperature, although why that did not affect the GPU is unclear).
Even with 100% load, the GPU temperature did not rise above the temperature for the light workload. This points to both good thermal management by Nvidia and HP, and likely that even with 100% load, the GPU silicon was still not being fully utilized, and there can be other computation loads which generate more heat due to the type of computation it performs.

Obviously many more trial would have to be taken for these numbers to be reliable, but I believe that for the price of the accessories, even this vague data is sufficient to suggest that there is a good return on investment. If future workloads are such that the temperatures are significantly higher than these measured, then additional cooling countermeasures would need to be taken, such as using fans that can move more air, or locating the laptop in the coolest part or an air-conditioned part of the house.

Final Decision

Time to bring down the hammer

Buying a laptop almost sight-unseen, without testing it in person, carries with it a degree of risk, but with the number of possibilities out there, personally testing even a fraction is rather difficult. So based on customer reviews and a roll of the dice, and after considering the above requirements, as well as prices, price/performance ratios, features, and discounts, I have settled on buying the following laptop online:

HP Omen 17
Intel Core i7-7700HQ (6 MB cache, 4 cores)
16 GB DDR4-2400 SDRAM (2 x 8 GB)
NVIDIA GeForce GTX 1070 (8 GB GDDR5 dedicated)
17.3" diagonal FHD IPS anti-glare WLED-backlit (1920 x 1080)
256GB NVMe M.2 SSD
1 TB 7200 rpm SATA HD (32MB cache)

Sorry, no banana for scale

The laptop is a monster of a machine, weighing in at about 3.35kg (7.4lbs). The power brick alone is about the size of a bento box. It has a full-size keyboard with a numeric keypad, which is pleasant enough to type on, although the key travel is a little shallow, and spacing a little off, so it's not as satisfying as some other laptops I've used. The performance is smooth, and there are plenty of ports and connectors for accessories. There are no glaring issues, and overall I am sufficiently satisfied with the machine.

Please follow along as I research which online courses to take in my next blog, In Which Instruction were Sought, Part 1 (link), and leave any comments or suggestions you have below. Thanks!

Oh, as a final note, I'm sure it will not come as a surprise for you to read that I named my laptop Still Waters. ^^

Still Waters Run Deep Learning