In Which Instructions were Sought, Part 1
 |
| "Give me six hours to chop down a tree and I will spend the first four sharpening the axe." - Abraham Lincoln |
Now that I have my machine ready, it is time to dive into the actual study of deep learning. (Please read my previous blog entries, In Which a Hardware Platform was Chosen, Part 1 (link) and Part 2 (link), where I talked about the decision between cloud and personal device, and between desktop and laptop computers, and the specs of the computer I will use for my study of deep learning.)
Nose in a Book... or Screen
 |
| Do birds even have a sense of smell? (Wikipedia link) |
Nowadays there are so many resources online and offline that you can find instructional material in a plethora of formats, whether you prefer prerecorded videos, live videos, in-person classes, online courses, e-books, dead tree books, audiobooks, etc. It's all out there. What would work best would depend on your preferences, and the way that you absorb material most naturally.
 | |||
|
Personally, classroom instruction works very well for me. Seeing and hearing about the topic in person, being able to ask questions and getting answers immediately is very valuable in tightening the feedback loop and reinforcing what I learned. However, the time, energy, and money commitment of an in-person course makes it non-ideal for me at this time.
At the other extreme, I have studied some topics through audiobooks, which I listened to during the commute to work. This method worked somewhat... Not being able to jot down any notes or my questions as soon as they popped into my mind means that my retention was low and my understanding of the topic was incomplete. However, for some lighter topics which doesn't involve much critical thinking, it works well enough.
A middle ground is some of courses offered by the major MOOC (massive open online course) providers such as Coursera (Coursera link), Udacity (Udacity link), and EdX (EdX link). This format provides video instructions and some mentoring, as well as a community of students who can discuss together any question or issues they may have. The MOOC providers also offer certificates of completion for certain courses, which can be added to resumes and LinkedIn profiles to show that you have gone through the course. Paying for the course is also a good way make myself more accountable, a self motivating trick to make me take the course more seriously, and to work quickly for those courses which are pay-by-the-month subscriptions.
But so as to not limit myself to any specific type of course, and keep my eyes open for quality instructions, I scoured various online forums including Reddit, Hacker News, Quora, and some blogs to get a list of recommendations, then went through them to pick out the best of the best across several categories.
It's Good to Be on Top
 |
| No time for losers 'cause they are the champions |
Here are the courses that I have found to have a comprehensive curriculum and/or widespread acclaim. In no particular order:
____________________________________________________________
Coursera Deep Learning Specialization (Coursera link)
Duration: 3 months (11 hours a week)
Cost: $49/month
Feedback: Online discussion forum with students and mentors
One of the most popular and highly recommended courses out there, Professor Andrew Ng (Wikipedia link) from Stanford University has taught this material for a long time and in many formats. You can find the videos where he first taught machine learning at Stanford back in 2008 on YouTube (YouTube link), and the subsequent Coursera version (Coursera link) after he co-founded Coursera in 2011. Andrew Ng has been credit with helping to popularize the use of deep learning, through his teachings and his work at Google and Baidu.
The cost is rather low for the amount of material you get. But if it is still a concern, the courses are available for free. You won't get the assignment grading or an official certificate of completion, but can still access all the videos and assignments as well as the forum.
____________________________________________________________
Duration: 14 weeks (10 hours a week)
Cost: Free
Feedback: Online Discourse and Slack discussion forums with students
Lauded by many who took the fast.ai course as being a more approachable way of studying deep learning with a bottom-up approach. Instead of starting with high level theory, the fast.ai courses instead dives right into using neural network, starting from running practical examples and then working you way up to an understanding of what a neural network is. This is similar to learning to drive a car before learning the how to build or designing a car, and depending on your goals, whether it is to design, build, or use neural networks, this method may resonate more with you. fast.ai was co-founded by Jeremy Howard, a faculty member at University of San Francisco. He was also the president and chief scientist at Kaggle, which is a platform for data scientists and machine learning practitioners to compete in machine learning competitions and to share code.
____________________________________________________________
Stanford CS231n: Convolutional Neural Networks for Visual Recognition (course material: GitHub link, videos: YouTube link), plus Stanford CS224n: Natural Language Processing with Deep Learning (course material: Stanford link, video: YouTube link)
Duration: 35 videos (44 hours), plus time for assignments
Cost: Free
Feedback: None
If you prefer a more rigorous, university-style education, these two courses would be right up your alley. Being Stanford University courses, there is a higher expectation of not only familiarity, but ease and comfort with understanding and using the math and programming concepts that are assumed to be basic knowledge in the course. If you are weak in these areas, you may need to either take some refresher courses beforehand, or be willing to stop in the middle of the course and embark on a tangent course in order to fully understand what's being taught. Being that these are videos on YouTube, you can download them and watch them whenever and wherever, without having to worry about bandwidth or connectivity, although I would recommend watching these in an environment where you can concentrate, for better absorption of the material. You would need your main machine, or internet connectivity if you will be using cloud compute services, when you do your assignments though (... you are planning to do the assignments, right? It makes the concepts solidify much more thoroughly, and lets you figure out which parts you truly understood).
____________________________________________________________
Udacity Deep Learning Nanodegree (Udacity link)
Duration: 4 months (12 hours a week)
Cost: $399/month or $1436 for 4 months
Feedback: Student forum and individual 1-on-1 mentoring
For a more full-service MOOC, Udacity offers their nanodegree programs which provide not only the discussion forums that other MOOCs offer, but also personalized feedback from a project reviewer, a 1-on-1 technical mentor, as well as career coaching. This comes closest to being a university learning experience without having to actually be in the classroom physically at a certain time. These extra services are reflected in the cost, which can be quite high if you are just taking these courses for enjoyment or as a hobby. Udacity has many courses which can be taken at no cost, but the nanodegree programs are interactive, and are not available for free.
____________________________________________________________
These four choices cover the gamut of price from free to over a thousand dollars, and feedback from none to 1-on-1 mentoring, which are what I think are two major differentiating points between the many resources out there. I can't speak directly to the quality of these choices yet, but based on the comments from various online discussions, these should all be well worth someone's time to study.
Diving In
 |
| What kind of bandwidth do you think I can get on that? |
Now it's my turn to do the studying. I will return once I have taken some of these courses, and provide a summary and a review of the material. The next blog entry is In Which Instructions were Sought, Part 2 (link). See you all in a few months!
Comments
Post a Comment