The Chatbot That's Acing the Largest Turing test in History
Meet Xiaoice. She’s empathic, caring, and always available — just not human.
February 4, 2016 at 04:20PM
via Digg http://ift.tt/1nINJA2
Posted by EricEnge
Machine learning is already a very big deal. It's here, and it's in use in far more businesses than you might suspect. A few months back, I decided to take a deep dive into this topic to learn more about it. In today's post, I'll dive into a certain amount of technical detail about how it works, but I also plan to discuss its practical impact on SEO and digital marketing.
For reference, check out Rand Fishkin's presentation about how we've entered into a two-algorithm world. Rand addresses the impact of machine learning on search and SEO in detail in that presentation, and how it influences SEO. I'll talk more about that again later.
For fun, I'll also include a tool that allows you to predict your chances of getting a retweet based on a number of things: your Followerwonk Social Authority, whether you include images, hashtags, and several other similar factors. I call this tool the Twitter Engagement Predictor (TEP). To build the TEP, I created and trained a neural network. The tool will accept input from you, and then use the neural network to predict your chances of getting an RT.
The TEP leverages the data from a study I published in December 2014 on Twitter engagement, where we reviewed information from 1.9M original tweets (as opposed to RTs and favorites) to see what factors most improved the chances of getting a retweet.
I got my first meaningful glimpse of machine learning back in 2011 when I interviewed Google's Peter Norvig, and he told me how Google had used it to teach Google Translate.
Basically, they looked at all the language translations they could find across the web and learned from them. This is a very intense and complicated example of machine learning, and Google had deployed it by 2011. Suffice it to say that all the major market players — such as Google, Apple, Microsoft, and Facebook — already leverage machine learning in many interesting ways.
Back in November, when I decided I wanted to learn more about the topic, I started doing a variety of searches of articles to read online. It wasn't long before I stumbled upon this great course on machine learning on Coursera. It's taught by Andrew Ng of Stanford University, and it provides an awesome, in-depth look at the basics of machine learning.
Warning: This course is long (19 total sections with an average of more than one hour of video each). It also requires an understanding of calculus to get through the math. In the course, you'll be immersed in math from start to finish. But the point is this: If you have the math background, and the determination, you can take a free online course to get started with this stuff.
In addition, Ng walks you through many programming examples using a language called Octave. You can then take what you've learned and create your own machine learning programs. This is exactly what I have done in the example program included below.
First of all, let me be clear: this process didn't make me a leading expert on this topic. However, I've learned enough to provide you with a serviceable intro to some key concepts. You can break machine learning into two classes: supervised and unsupervised. First, I'll take a look at supervised machine learning.
At its most basic level, you can think of supervised machine learning as creating a series of equations to fit a known set of data. Let's say you want an algorithm to predict housing prices (an example that Ng uses frequently in the Coursera classes). You might get some data that looks like this (note that the data is totally made up):
In this example, we have (fictitious) historical data that indicates the price of a house based on its size. As you can see, the price tends to go up as house size goes up, but the data does not fit into a straight line. However, you can calculate a straight line that fits the data pretty well, and that line might look like this:
This line can then be used to predict the pricing for new houses. We treat the size of the house as the "input" to the algorithm and the predicted price as the "output." For example, if you have a house that is 2600 square feet, the price looks like it would be about $xxxK ?????? dollars.
However, this model turns out to be a bit simplistic. There are other factors that can play into housing prices, such as the total rooms, number of bedrooms, number of bathrooms, and lot size. Based on this, you could build a slightly more complicated model, with a table of data similar to this one:
Already you can see that a simple straight line will not do, as you'll have to assign weights to each factor to come up with a housing price prediction. Perhaps the biggest factors are house size and lot size, but rooms, bedrooms, and bathrooms all deserve some weight as well (all of these would be considered new "inputs").
Even now, we're still being quite simplistic. Another huge factor in housing prices is location. Pricing in Seattle, WA is different than it is in Galveston, TX. Once you attempt to build this algorithm on a national scale, using location as an additional input, you can see that it starts to become a very complex problem.
You can use machine learning techniques to solve any of these three types of problems. In each of these examples, you'd assemble a large data set of examples, which can be called training examples, and run a set of programs to design an algorithm to fit the data. This allows you to submit new inputs and use the algorithm to predict the output (the price, in this case). Using training examples like this is what's referred to as "supervised machine learning."
This a special class of problems where the goal is to predict specific outcomes. For example, imagine we want to predict the chances that a newborn baby will grow to be at least 6 feet tall. You could imagine that inputs might be as follows:
The output of this algorithm might be a 0 if the person was going to shorter than 6 feet tall, or 1 if they were going to be 6 feet or taller. What makes it a classification problem is that you are putting the input items into one specific class or another. For the height prediction problem as I described it, we are not trying to guess the precise height, but a simple over/under 6 feet prediction.
Some examples of more complex classifying problems are handwriting recognition (recognizing characters) and identifying spam email.
Unsupervised machine learning is used in situations where you don't have training examples. Basically, you want to try and determine how to recognize groups of objects with similar properties. For example, you may have data that looks like this:
The algorithm will then attempt to analyze this data and find out how to group them together based on common characteristics. Perhaps in this example, all of the red "x" points in the following chart share similar attributes:
However, the algorithm may have trouble recognizing outlier points, and may group the data more like this:
What the algorithm has done is find natural groupings within the data, but unlike supervised learning, it had to determine the features that define each group. One industry example of unsupervised learning is Google News. For example, look at the following screen shot:
You can see that the main news story is about Iran holding 10 US sailors, but there are also related news stories shown from Reuters and Bloomberg (circled in red). The grouping of these related stories is an unsupervised machine learning problem, where the algorithm learns to group these items together.
A great example of a machine learning algo is the Author Extraction algorithm that Moz has built into their Moz Content tool. You can read more about that algorithm here. The referenced article outlines in detail the unique challenges that Moz faced in solving that problem, as well as how they went about solving it.
As for Stone Temple Consulting's Twitter Engagement Predictor, this is built on a neural network. A sample screen for this program can be seen here:
The program makes a binary prediction as to whether you'll get a retweet or not, and then provides you with a percentage probability for that prediction being true.
For those who are interested in the gory details, the neural network configuration I used was six input units, fifteen hidden units, and two output units. The algorithm used one million training examples and two hundred training iterations. The training process required just under 45 billion calculations.
One thing that made this exercise interesting is that there are many conflicting data points in the raw data. Here's an example of what I mean:
What this shows is the data for people with Followerwonk Social Authority between 0 and 9, and a tweet with no images, no URLs, no @mentions of other users, two hashtags, and between zero and 40 characters. We had 1156 examples of such tweets that did not get a retweet, and 17 that did.
The most desirable outcome for the resulting algorithm is to predict that these tweets not get a retweet, so that would make it wrong 1.4% of the time (17 times out of 1173). Note that the resulting neural network assesses the probability of getting a retweet at 2.1%.
I did a calculation to tabulate how many of these cases existed. I found that we had 102,045 individual training examples where it was desirable to make the wrong prediction, or for just slightly over 10% of all our training data. What this means is that the best the neural network will be able to do is make the right prediction just under 90% of the time.
I also ran two other sets of data (470K and 473K samples in size) through the trained network to see the accuracy level of the TEP. I found that it was 81% accurate in its absolute (yes/no) prediction of the chance of getting a retweet. Bearing in mind that those also had approximately 10% of the samples where making the wrong prediction is the right thing to do, that's not bad! And, of course, that's why I show the percentage probability of a retweet, rather than a simple yes/no response.
Try the predictor yourself and let me know what you think! (You can discover your Social Authority by heading to Followerwonk and following these quick steps.) Mind you, this was simply an exercise for me to learn how to build out a neural network, so I recognize the limited utility of what the tool does — no need to give me that feedback ;->.
So now that we know a bit more about what machine learning is about, let's dive into things that Google may be using machine learning for already:
One approach to implementing Penguin would be to identify a set of link characteristics that could potentially be an indicator of a bad link, such as these:
Note that any one of these things isn't necessarily inherently bad for an individual link, but the algorithm might start to flag sites if a significant portion of all of the links pointing to a given site have some combination of these attributes.
What I outlined above would be a supervised machine learning approach where you train the algorithm with known bad and good links (or sites) that have been identified over the years. Once the algo is trained, you would then run other link examples through it to calculate the probability that each one is a bad link. Based on the percentage of links (and/or total PageRank) coming from bad links, you could then make a decision to lower the site's rankings, or not.
Another approach to this same problem would be to start with a database of known good links and bad links, and then have the algorithm automatically determine the characteristics (or features) of those links. These features would probably include factors that humans may not have considered on their own.
Now that you've seen the Penguin example, this one should be a bit easier to think about. Here are some things that might be features of sites with poor-quality content:
Once again, you could start with a known set of good sites and bad sites (from a content perspective) and design an algorithm to determine the common characteristics of those sites.
As with the Penguin discussion above, I'm in no way representing that these are all parts of Panda — they're just meant to illustrate the overall concept of how it might work.
The key to understanding the impact of machine learning on SEO is understanding what Google (and other search engines) want to use it for. A key insight is that there's a strong correlation between Google providing high-quality search results and the revenue they get from their ads.
Back in 2009, Bing and Google performed some tests that showed how even introducing small delays into their search results significantly impacted user satisfaction. In addition, those results showed that with lower satisfaction came fewer clicks and lower revenues:
The reason behind this is simple. Google has other sources of competition, and this goes well beyond Bing. Texting friends for their input is one form of competition. So are Facebook, Apple/Siri, and Amazon. Alternative sources of information and answers exist for users, and they are working to improve the quality of what they offer every day. So must Google.
I've already suggested that machine learning may be a part of Panda and Penguin, and it may well be a part of the "Search Quality" algorithm. And there are likely many more of these types of algorithms to come.
Given that higher user satisfaction is of critical importance to Google, it means that content quality and user satisfaction with the content of your pages must now be treated by you as an SEO ranking factor. You're going to need to measure it, and steadily improve it over time. Some questions to ask yourself include:
There are many ways that Google can measure how good your page is, and use that to impact rankings. Here are some of them:
For more on how content quality and user satisfaction has become a core SEO factor, please check out the following:
Machine learning is becoming highly prevalent. The barrier to learning basic algorithms is largely gone. All the major players in the tech industry are leveraging it in some manner. Here's a little bit on what Facebook is doing, and machine learning hiring at Apple. Others are offering platforms to make implementing machine learning easier, such as Microsoft and Amazon.
For people involved in SEO and digital marketing, you can expect that these major players are going to get better and better at leveraging these algorithms to help them meet their goals. That's why it will be of critical importance to tune your strategies to align with the goals of those organizations.
In the case of SEO, machine learning will steadily increase the importance of content quality and user experience over time. For you, that makes it time to get on board and make these factors a key part of your overall SEO strategy.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Posted by JayLeary
When we launched Moz Content at the end of November, we limited the subscription to a single tier. At the time we wanted to get the product out in the wild and highlight the importance of content auditing, competitive research, and data-driven content strategy. To all of those early adopters who signed up as Strategists over the last few months, a big thanks.
Since then, we’ve made a long list of improvements to both Audit performance/stability and the size of our Content Search index. Along with those updates we also added two new subscriptions for larger sites and agencies handling multiple clients: Teams and Agencies. The new tiers not only increase page and Audit limits, but also enable Google Analytics integration with Tracked Audits.
For those that aren’t familiar, Tracked Audits let you trend and monitor the performance of your site’s content over time. On the backend, Moz Content re-audits your site every week in order to discover new pages and update site metrics. This allows you to compare, say, the average shares per article across your entire site from week-to-week or month-to-month.
To date, Moz Content Audits have focused on links and shares. We did that intentionally, since a major goal for the product was to enable the analysis of any site on the web, including competitors. We also wanted to give agencies the freedom to prospect or audit clients without painful integrations or code snippets.
That said (and I’m guessing you’ll agree), figuring out your content ROI usually requires a deeper look at site performance.
With Moz Content, we stopped short of a full, conversion-focused analysis suite with custom tracking code and the like. Instead we’re focused on a product that delivers content analysis and insights to our entire community of online marketers — not just the select few that can afford it. Besides, there are plenty of big-ticket tools out there filling the niche, great products like Newscred, Kapost, SimpleReach, and Idio.
So while we’re not jumping into the “enterprise” ring (just yet), we did want to give data-driven marketers a leg up in their analysis. The good news: if you’re a Teams or Agencies subscriber, you now have the added option of integrating Google Analytics with Tracked Audits.
It’s easy to get started, and if you’re a Moz Pro subscriber you’re probably familiar with the authentication flow. Just go to any Tracked Audit you have GA data for, scroll down to the middle of the report, and click “Connect Google Analytics”:
Once you’ve connected a profile, Moz Content immediately pulls in key metrics for each URL in the Inventory:
You’ll probably notice we’re only displaying page views in the inventory. While we didn’t have the real estate to include all of the metrics we’re collecting in the interface, we’ve added them to the CSV export:
Once you dump the data to Excel you’ll see the following metrics for the Organic and Paid segments, as well as a rollup of all referrers:
After we’ve collected multiple audits, Moz Content also starts trending aggregated metrics so you can get a sense of performance over time:
We’re hoping this added reporting gets you a step closer to that all-important ROI analysis. Some of you will already have a sense of how much a page view is worth, or the impact of a unique page view on a specific conversion. And for the Google Analytics pros out there, properly configuring Page Value will give you a direct indicator of a page’s effectiveness.
This is just the beginning, and we’d love to hear about other ways we could make the GA data more useful. Please reach out and let us know what you think by clicking the round, blue Help button in the lower-right corner of the Moz Content app.
We’re pushing out code every day to improve the app experience and build on the current features. We’re also growing our Content Search index with new sources of popular, trending content. If you haven’t tried it for a while, we encourage you to take a second look. Head over to http://moz.com/content and start a search or enter a domain to preview an Audit (be sure to sign into your community account to access both the Dashboard and increased limits).
On the feature front, we’re pushing to integrate Twitter data into both the Audit and Content Search. As I’m sure you’ve noticed, this has been missing in our reporting to date. While we won’t have exact share counts for individual articles (see Twitter’s decision to deprecate the share count endpoint), we’re confident we can provide related information that’ll be useful for your Twitter analysis.
As we develop this and other features, we’re always on the lookout for feedback from our community. As always, feel free to reach out to our Help team with any issues or feedback about the product. And if you’ve used Moz Content and are interested in beta testing the latest, shoot us an email at mozcontent+testing@moz.com and we’ll add you to the list.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Posted by Isla_McKetta
Think you're up on the latest developments in local SEO?
One year ago we asked you to test your local SEO knowledge with the Local Search Expert Quiz. Because the SERPs are changing so fast and (according to our latest Industry Survey) over 42% of online marketers report spending more time on local search in the past 12 months, we've created an updated version.
Written by local search expert Miriam Ellis, the quiz contains 40 questions designed to test both your general local SEO knowledge and your industry awareness. Bonus? The quiz takes less than 10 minutes to complete.
Ready to get started? When you are finished, we'll automatically score your quiz.
Although the Local Search Expert Quiz is just for fun, we've established the following guidelines for bragging rights:
Didn't get the score you hoped for? Brush up on your local SEO knowledge with this collection of free learning resources:
You can also learn the latest local search tips and tricks by signing up for the MozCon Local one-day conference, subscribing to the Moz Local Top 7 newsletter, or reading local SEO posts on the Moz Blog.
Don't forget to brag about your local search expertise in the comments below!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Posted by mattround
Before joining Distilled I worked for UsVsTh3m, an experimental Trinity Mirror project, where we created hundreds of games, quizzes and daft "toys." We had unprecedented freedom to try out new interactive formats, learning a great deal about what works… and what doesn’t.
The key to success was "viral" traffic. You’ve probably heard the term bandied about in reference to something popular, and might even have rolled your eyes; it’s a much-abused buzzword.
The idea is that online word-of-mouth can drive exponential traffic growth and broad media coverage with little or no traditional promotional support, but achieving this requires a certain way of thinking. This article focuses on interactive content, but many of the same principles will apply to other formats.
It’s useful to aim for interactive content to be…
You usually need all three aspects to be strong to get a viral hit. It’s easy to focus on one, an experienced team can usually achieve two, but it’s difficult to consistently get the full set.
Crudely, you can think of it in terms of losing potential sharers, ultimately needing to end up with more than one to start the next cycle(s). This image explains it nicely:
Congratulations, it’s going viral! That’s a massive simplification, but a helpful one.
Develop a concept that ties in with the player’s personal attributes: age, location, abilities, personality, etc. For example, measuring reaction time in milliseconds is fine… but if you can correlate it with age, then you’ve immediately got something far more compelling.
Reinforce a sense of belonging; tribes can be regional, generational, interest-based, political, etc. Perhaps play different tribes off against each other so that your interactive content can address niche groups while having broad appeal overall.
It tells you something about yourself or, more likely, confirms a flattering/intriguing attribute, leading into...
Sharing to make yourself look good… but without it seeming too blatant.
Ensure the player is hooked and will want others to share in that. Although bear in mind that the best games often aren’t the most viral — adding multiple levels and features to a game tends to put off non-gamers and can actually reduce sharing (enthusiasm has a chance to ebb away, and the game will tend to end on a low note when the player finally fails or quits).
People are impressed by fast-turnaround topical content, and sharing it can show you’re up-to-date (perhaps even the first in your social circle to discover something). We regularly developed and launched games in half a day at UsVsTh3m, and more than once within an hour. This obviously isn’t feasible for most commercial projects, but with more agile development and approval processes you can reduce lead times.
Overwhelm the senses: strong use of music, dance, animation, spectacular explosions, anything that’s a straightforward pleasure.
“Can you beat this score?”
“I got this result, how about you?” This is much "softer" than direct competition and typically more welcoming for a broad audience.
Things like global counters, polls, or territorial maps can create the sense of playing your part in a bigger cause. Even just clicking something to increment a number can be made hard to resist with the right "cause."
It’s still possible for something to succeed simply by being good, but in the absence of any other aspects it’d better be really good. Knock-your-socks-off good.
Of course, all of the above overlap and interrelate, and it’s by no means an exhaustive list.
Something that’s strongly viral can actually just be exposed to a few hundred people via Twitter or Facebook. It won’t need a big push; the viral mechanism will ensure it spreads and attracts media attention.
It’s often useful to accompany a launch with relevant press material, perhaps teasing out key angles or supplementary content/data to suit each type of media outlet. Don’t force a story if there isn’t one, though; you don’t want to jeopardize later coverage based around “this cool thing is going viral.”
If the stats are showing it’s strongly going viral (this should be obvious within minutes), you’re then in the fortunate position of planning for success. Keep an eye out for initial coverage that may benefit from additional material, and look to do a follow-up press campaign at a suitable milestone (e.g. at X million visitors, or when you have interesting data to share), broadening the coverage.
If it’s not going viral, stop and consider whether minor changes to wording might make it more clickable. Look at whether it needs to break into a niche audience or broaden its appeal, and retarget accordingly. Although Twitter drives far less traffic than Facebook, it offers more freedom to experiment, target influential individuals, and re-promote over time. If a topical angle may arise, perhaps wait and be ready to repackage and relaunch at a moment’s notice.
UsVsTh3m’s North-o-Meter (sadly, this is currently broken due to hosting issues) used multiple-choice questions to guesstimate how Northern/Southern you are. Despite being entirely UK-focused, within just 4 days of launch it had 3.6 million visitors, 1 million Likes, 1.1 million comments on Facebook, and 41,000 tweets. It went on to get millions more visits, virtually saturating the potential audience. Countless similar quizzes had used this topic before, so why did this one make such a big impact?
How Old Are Your Reactions?, produced by Distilled for JustPark, is a simple web game where you stop a car with a tap/click. Your reaction time is then used to look up the corresponding age for that score, based on a survey of 2,000 players.
Our thinking beforehand was that it would work well due to the following aspects:
This all seems to have been borne out by the stats since launch: Over 3 million unique page views, nearly 300,000 social shares, and links from over 400 domains.
Always ask yourself:
The way people share and interact is constantly changing, and reaching large audiences is always challenging, but the approaches I’ve outlined can help you to devise interactive content that’ll have a great shot at going viral.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!