I’ve had the fortune of working with several different businesses in a wide variety of departments including accounting, data science, risk analysis, product management, marketing, project management, and even a bit of recruiting. This is the first in a series of posts about different jobs and what makes them fun (and important!)
Let’s start with accounting. Making money is a key goal in any business-for most, it’s their primary goal. But figuring out whether any individual strategy, marketing campaign, or product makes money is actually rather difficult. The big customer who drives most of your revenue may be massively unprofitable after you account for commissions, chargebacks, and special demands. On the other hand a previously overlooked niche might turn out to have extremely high margins.
The accountants are responsible for bringing this information together. They take all the data for different types of costs and revenues, put it all together, and figure out where the money is coming from and where its going. This means accountants have the best grasp of how profitable or costly any initiative has been. That in turn means that accountants can provide strong, data-driven recommendations what a business should change in order to make more money.
And that’s part of what makes accounting so fun. By slicing the data in a different way and getting a better view of what’s going on, you can easily identify opportunities that can make a huge impact on the business. In my first bookkeeping job I was able to help a business recover more than half a million dollars of cash, identify & renegotiate with unprofitable customers, and find some underserved markets that were providing huge margins. How cool is that?
The vast majority of career advice simply sucks. Most advisors write empty platitudes such as “follow your passion” without daring to make any specific claims, and their advice ends up helping no one or actually being counterproductive. So here are some observations on what I’ve seen work, at least for 20-somethings:
Most Machine Learning algorithms today rely on a process known as Stochastic Gradient Descent, or SGD. SGD is an iterative process based on the idea of minimizing a cost function. Roughly speaking, at each step SGD finds the direction to move your weights that most decreases your cost function, then moves your weights by a certain amount in that direction.
That “certain amount” is known as the learning rate, and it’s a key parameter whenever you’re training an algorithm on a large dataset. Set the learning rate too high and your algorithm simply won’t converge, or you’ll suffer from very high bias. Set the learning rate too low, and your algorithm will take forever to train.
The most commonly suggested approach is to start with a high learning rate that decreases over time. You might have an if statement that drops the learning rate by 20% if your model goes through a few iterations where the cost doesn’t drop.
The problem with a purely decreasing learning rate is that you often only need a small learning rate for a few iterations. So a decreasing learning rate ends up being very inefficient. A better method is to have a learning rate that increases by a small amount after each iteration where the cost goes down, and drops sharply any time the cost goes up.
This can easily make your algorithm train 10 times as quickly. For example, I ran one algorithm with 10,000 iterations and a dynamic learning rate starting at .02. After 1,000 iterations the learning rate was at .06. From 1,000-1,100 the learning rate dropped to about .01. Then the learning rate went back up to .12 by iteration 2,000 and steadily increased until stopping at 10,000 with a final cost of .32. Using a static or decreasing-only learning rate, I had to train to 100,000 iterations to get the cost down to .38!
So next time you’re training a Neural Network or other SGD-based algorithm, consider using a dynamically increasing learning rate. It just might turn an overnight task into something that takes half an hour.
Categories: Machine Learning
“When the state information technology chief of Massachusetts announced in 2005 that the state was going to standardize its computer document format to be more compatible with open-source software-software that isn’t controlled by a company-than with Microsoft products, loud choruses of support and outrage ensued, closely followed by the announcement of an investigation into the chief’s apparently perfectly legal state-funded travel for participation in open-source software conferences, followed by a new state-government announcement that it would work with Microsoft to stay compatible with the company’s document formats, followed by an announcement from Microsoft that it would bring its document format closer to an open-source format.”-A Perfect Mess: The Hidden Benefits of Disorder