What was the last book you truly enjoyed? You know the one that had you hooked and you could not put down until you were done. What did you like the most about it? Now imagine being the publisher. Do you think it would have been easy for them to judge the book? Sometimes, the most unexpected books become bestsellers. J.K. Rowling was turned down by 12 publishers and we all know how that turned out.
So, what guides publishers decisions? Is it the storyline, characters or the writer’s ability to keep the reader engaged? As we move forward with new technology, there are now algorithms that could help publishers identify the next bestseller.
In this book summary readers will discover:
- Why predicting a bestseller is hard
- The bestseller-ometer
- The elements needed for a successful book
- Why female authors are better
Key lesson one: Why predicting a bestseller is hard
Do you know how many books are published annually? Considering fiction alone, the number is around 50 000 and that excludes e-books. From these books, about 200 will make the New York Times bestseller list. That is 0.4 per cent. So what makes these books so special?
Well, that question seems to have multiple answers. It is not easy to predict which book will make it to the top. In fact, from the time the first best-selling book list was published in 1891, it was very clear that what is popular and what is considered quality writing are two very different things. It became pretty evident over time that books that were considered to be badly written often turned out to be popular. This has baffled critics for decades. In recent years, books like Fifty Shades of Grey, The Da Vinci Code and The Girl with the Dragon Tattoo have all gained success to the surprise of critics worldwide. They were all picked on because of their weak characters and mixed up plots.
Even though it seems that publishers can assume that poorly written books will make the bestseller lists, exactly which poorly written books do they choose? Predicting bestsellers is a difficult job. Therefore there needs to be a way to figure out what all these books have in common.
Key lesson two: The bestseller-ometer
In 2010, science came to the rescue. Authors began to study best-selling novels and tried to identify what made them best-sellers. Over the course of five years, they were able to identify consistent patterns which ultimately were so robust, an algorithm could be designed.
This algorithm was called the bestseller-ometer and was proven to be highly accurate when tested. The algorithm was able to successfully predict 80 to 90 per cent of the bestsellers that ended up on the New York Times bestseller list. They also tested it by using previous bestsellers but leaving the anonymous so that it would ignore the author’s current status. It continued to correctly predict bestsellers 85 per cent of the time. Even with being wrong 15 per cent of the time, this algorithm has the potential to be a huge asset to the publishing world. It correctly predicted that Harry Potter had a 95 per cent chance of being a best seller. That would have silenced the sceptics and put Harry Potter on the shelves a lot faster.
But besides a tool for publishers, authors could also benefit from the bestseller-ometer – especially first-time authors. Getting your first book out is always a struggle, but the bestseller-ometer could possibly give authors the guidance they need to get it right.
Key lesson three: The elements needed for a successful book
The algorithm considers a number of things when working out if a book will be a bestseller. The most important of these factors is the topic. This is not to be confused with the genre of the book. The topic refers to the themes contained in the book. Love and crime, for example, is a topic that is popular no matter the genre of the book.
Looking at the algorithm in action, the example is given of Jodi Picoult’s book, House Rules. The book is a family drama about a kid with Asperger’s that gets accused of murder. The topics identified in the book and listed in order of percentage were kids, crime, legal settings, domestic situations and close relationships. The dominant topic was kids but the fact that crime and close relationships were also present, albeit in the background, made the algorithm predict that the book would be a bestseller.
The bestseller-ometer has the ability to determine these topics by studying each word and the context it is used in. This distinction is important as the same word can have very different meanings. The process it uses is called topic modelling and is what the algorithm uses to determine the topics and add up their proportions. To date, the most successful topic is crime and despite everyone thinking that sex sells, sex remains one of the lowest-performing topics.
After the topic, the next important factor is the use of emotion in the plot. The best books take us on an emotional rollercoaster. One of the books which surprised everyone with its success was Fifty Shades of Grey. It received horrendous reviews but despite that provided readers with big emotions. When analysed by the algorithm, if only the writing style were to be taken into consideration, the result would have been a 50 per cent chance at success. However, because the emotional content and the topic were taken into consideration, it resulted in a 90 per cent chance at success. The algorithm identified the main topic not as sex but as an intimate human relationship with little conflict. This is exactly what made the book so popular and successful. The only other book that achieved the same pattern of emotional ups and downs was The Da Vinci Code. Two very different books but with the same emotional chart.
Lastly, the writing style of the author also greatly contributes to the success of a novel. The algorithm proved useful in two ways when it came to writing style. Firstly, it proved that simple writing is a much better way of gaining success than fancy writing with a lot of complicated phrases. It established this by analysing common words and measures their use against other successful books. For example, successful books use the word do two times more than unsuccessful books. The sentence structure is also concise and contains a few adjectives and adverbs. Secondly, the algorithm can also identify a writer’s unique writing style. When The Cuckoo’s Calling came out in 2013, rumours circulated that the author was not Robert Galbraith. When the book was ran through the algorithm it correctly identified J.K Roeling as the author – she tried out a new genre and used a pen name. This proved that some writers have a strong and unique writing style that although unidentifiable by a person, can be identified by a computer.
In combination, these factors are all analyzed by the bestseller-ometer to determine a book’s success. They all contribute in their own ways as described above but have provided pretty accurate results when used.
Key lesson four: Why female authors are better
There is one last thing that the algorithm has identified which is very interesting. In all the books analysed it observed a trend that showed that female authors are rated much higher than male authors when it came to the style of the book. It did not come up when analysing the plot or theme of the book on their own, only when the book’s overall style.
They investigated this further by considering debut novels, but once again 90 per cent of the successful books were written by females. Even more interesting was that the algorithm could determine the gender of the author by analysing a novel. This led the developers of the algorithm to determine what exactly it was basing its results on. To do this they considered the books it got wrong. James Patterson’s books Suzanne’s Diary for Nicholas and Four Blind Mice were identified as being written by a female. Likewise, Toni Morrison was wrongfully identified as a male author.
It was then discovered that the author’s writing style contributed to this. Male authors tend to have what is known as a more sophisticated literary style whilst most female authors have a journalistic background. The females have the upper hand here as journalism teaches one to write simply and concisely which is important for a bestseller. In Patterson’s case, the reason he was identified as a female author by the algorithm is most likely due to his background in advertising. His writing style has a broader appeal.
The key takeaway from The Bestseller Code is:
Publishers can never really tell when a book will be a bestseller but after some careful analysis, an algorithm was developed which could help them do it. Bestsellers have more in common than we could recognize, but it proved easy for a computer once it knew what it was comparing and why. These patterns can accurately determine if a book will be a bestseller or not with 85 per cent accuracy. It takes into consideration the topic of the book, the writing style of the author and the emotions it evokes. The algorithm also other interesting implementations that make it an excellent tool for publishers and authors alike.
How can I implement the lesson learned in The Bestseller Code:
Now that you know what the bestseller-ometer looks for in a book, why don’t you try identifying them in a book that you are reading? If it is not on the bestseller list, compare it to one that is or vice versa. Can you spot the differences?
