The Top 10 Big Data Challenges: 2 Technological vs 8 Cultural

The Top 10 Big Data Challenges [Infographic].

This week’s infographic looks at the top 10 challenges organizations face in capitalizing on Big Data, courtesy ofTata Consultancy Services.

big-data-infographic

tweet infographic

The Top 10 Big Data Challenges

1. Cutural: Getting business units to share information across organizational silos

2. Technological: Being able to handle the large volume, velocity and variety of Big Data

3. Cultural: Determining what data, both structured and unstructured, and internal and external) to use for different business decisions

4. Cultural: Building high levels of trust between the data scientists who present insights on Big Data and the functional managers

5. Cultural: Finding and hiring data scientists who can manage large amounts of structured and unstructured data, and create insights

6. Cultural: Getting top management in the company to approve investments in Big Data and its related investments (e.g., training)

7. Technological: Putting our analysis of Big Data in a presentable form for making decisions (e.g., visualization/visual models)

8. Cultural: Finding the optimal way to organize Big Data activities in our company

9. Cultural: Understanding where int he company we should focus our Big Data investments

10. Cultural: Determining what to do with the insights that are created from Big Data

Comment SFR repère ses clients voulant le quitter grâce au Big Data

Comment SFR repère ses clients voulant le quitter grâce au Big Data.

Technologie : L’opérateur au carré rouge analyse le comportement de ses clients en ligne afin d’anticiper d’éventuelles envies d’aller voir ailleurs. La méthode serait efficace puisque les trois-quarts des abonnés contactés renoncent à se désabonner.

Si le Big Data occupe l’esprit de très nombreuses directions informatiques, force est de constater que le retour sur investissement tarde à se concrétiser. Une étude récente de Capgemini Consulting montre en effet les initiatives Big Data ne sont que dans 27% des cas qualifiées de réussites. S’y ajoutent 8% qui n’hésitent pas à parler de franc succès. Pour les autres, les attentes n’ont donc pas été satisfaites.

Du côté des opérateurs télécoms, l’équation semble un peu différente. On a évoqué ici les initiatives d’Orange Business Services qui monétise les données anonymisées de ses clients pour réaliser des produits à destination des spécialistes du tourisme par exemple. Des produits qui commencent à rencontrer un vrai succès.

Chez SFR, on va plus loin encore : l’analyse de données lui permet de faire baisser son taux de churn, c’est-à-dire le poucentage de clients le quittant pour aller chez la concurrence. Le churn est le cauchemar des opérateurs qui cherchent logiquement à retenir le plus possible leurs abonnés.

81% des candidats au départ repérés

C’est au détour d’un dossier compilant de nombreux retours d’expérience de Big Data dans les entreprises, que l’on découvre cette approche, fournie par Sinequa. L’idée est simple : il s’agit d’analyser (donc de surveiller ?) le comportement de ses clients en ligne afin d’anticiper d’éventuelles envies d’aller voir ailleurs.

“Quotidiennement, nous insérons dans Sinequa 20 millions de lignes de logs issus du trafic sur sfr.fr et m.sfr.fr, la version mobile”, précise Olivier Denti, en charge du premier projet Sinequa chez SFR. Le prestataire fournit ensuite des informations ciblées, par exemple sur la durée de la visite, le nombre de pages vues, les domaines visités (Assistance, Mail, Mobiles…), les produits consultés, les mots-clés saisis sur un moteur de recherche Internet ou sur sfr.fr, etc”.

De quoi permettre de repérer ceux qui auraient la tentation de partir, c’est à dire ceux qui consultent beaucoup de pages liées aux résiliations de contrat ou aux offres concurrentes…

Cette surveillance proactive pourrait choquer mais pour SFR, le retour sur investissement est plus que satisfaisant puisque cette méthode permet de mieux repérer les candidats au départ (81% affirme Sinequa) et de les appeler au bon moment, avant qu’ils aient pris leur décision. Et ça marche puisqu’un test mené sur un échantillon de ces personnes a révélé que 75% des clients contactés avant une résiliation restent chez SFR. Soit une économie de “millions d’euros” pour SFR, selon le spécialiste.

Contacté par nos soins SFR ne fait “aucun” commentaire. On peut néanmoins estimer que les concurrents de SFR appliquent les mêmes méthodes pour mieux connaître leurs clients…

Havas Media Belgique multiplie son expertise « Data »

Havas Media Belgique concrétise son ambition d’être l’agence media « 100% data driven ». A cette fin, John Greca intègre le département Data, Insight & Strategy (2MV) dirigé par Yves Wémers en tant que Data Manager.

John est fort de 10 ans d’expérience dans le monde digital, en agence (DigitasLBI, These Days, KBMG) et chez l’annonceur (Fnac, Mobistar et Belgacom/Proximus). Sa mission consiste à traduire les données digitales en outils d’amélioration du chiffre d’affaires au travers de la mesure de l’expérience client.

Il bénéficie de l’aide de Jessica Michotte (en charge de la gestion de la Data Management Platerform d’Havas Media Belgique : Artemis) et d’un data scientist (à l’arrivée imminente). Sophie Alderweireldt (ex-RMB et CIM) et Vanessa Sanctorum (ex-TNS et Sanoma), toutes deux Insights Experts, continueront à analyser les marques, les marchés et les consommateurs.

La mission de John et de son équipe vise à une meilleure compréhension des cycles d’achats des consommateurs et de l’influence de l’ensemble des canaux de communications sur ceux-ci. Les synergies avec toutes les entités digitales d’Havas Media Belgique (Affiperf, Socialyse et Ecselis respectivement pour le programmatic, le social media et la performance) offrent l’opportunité de qualifier les données produites par les différents canaux. Ces données sont ensuite utilisées dans les différents scénarios de « programmatic buying ».

HavasMediaBelgique_2MV

John Greca (Data Manager) : « Je suis particulièrement heureux de rejoindre Havas Media Belgique où je vois à quel point la volonté d’intégration des différents canaux et le dynamisme vont me permettre de pousser la réflexion Data encore plus loin. »

Hugues Rey (CEO Havas Media Belgique) : « Avec l’arrivée de John nous réalisons notre volonté d’intégrer les effets de l’ensemble des canaux de communications au travers du traitement des données. Cette évolution nous rapproche encore davantage des réalités business de nos clients et nous permet une réactivité accrue. »

Mobile analytics: Extending the reach of customer service to mobile | ITProPortal.com

5 reasons 2015 will be the year of Big Data | ITProPortal.com.

Big data has been one of the biggest trends over the last couple of years. Yet while companies seem to have gained a better understanding of the concept in 2014, there is still confusion about how to unlock its true business potential.

In 2015, I expect to see companies explore, and get to grips with this in a variety of areas. Some of my key predictions for the year ahead include:

SEE ALSO: 6 Google tricks that you probably never knew

1. Security Analytics: The hot topic for fraud detection

Analytics will become a key tool in detecting and preventing advanced threats in 2015. According to Mandiant’s M-Trends report on IT security, attackers spend around 229 days on a victim’s network before they are discovered, almost always using valid credentials, and 67 per cent of victims are notified about the threat by someone outside of their organisation.

As the adoption of connected devices grows and data becomes more interlinked, threats and their ability to spread quickly will be more pronounced. Data analytics will therefore be used more proactively to spot unusual event patterns, or anticipate what these patterns might be and set up alerts to escalate them to the right people so they can be solved before they make a major impact.

The fast moving nature of fraud does however mean that patterns are constantly changing and new ones will emerge, so writing rules into software simply isn’t enough. IT and security teams will need challenge themselves to constantly ask new questions of their data and ‘think like a criminal’ about how they would breach a system.

 2. Hadoop: From data store to valuable data insight

Despite accelerated adoption of Hadoop in EMEA in recent years, driving value from the data stored in Hadoop is a time consuming and expensive process that requires experienced data scientists. In 2015 however, analytics on Hadoop and elsewhere will become easier to use and accessible to anyone in a business regardless of their job role and technical know-how.

Self-service analytics, will mean that anyone within an organisation will be able to gain business insight from Hadoop in real-time, opening an organisation’s data to an entirely new audience.

The number and type of organisations looking to “test the waters” with Hadoop will also increase as managed/cloud services make this an affordable option.  The rise of PAYG (pay as you go) pricing plans with providers such as AWS, for example, means the initial investment in terms of software, infrastructure and skills can be minimised.

Companies will therefore have the freedom to experiment with Hadoop through “Big Data as a Service” to demonstrate ROI and evaluate the possible return of a bigger investment. This could also apply to departments within an organisation that opt for a DIY approach rather than relying on IT teams.

big data, hilarious, 2015 predictions

3. IoT: An evolution from “connected toothbrushes and Fitbits” to industrial data

In 2015, the conversation around IoT will extend beyond consumer devices to the disruption in traditional ‘bricks & mortar’ industries like building, manufacturing and transportation.

Manufacturing, for example, is increasingly benefitting from the combination of IoT and big data. By linking up sensors and robotics to automate processes, manufacturers are becoming more efficient, while also generating a massive amount of ‘machine data’ which can be indexed, monitored and analysed to provide real time problem solving, machine health monitoring and cost avoidance.

An example in transport is New York Air Brake which is using Splunk Enterprise to save up to $1 billion in fuel and other costs on U.S. railroads. As a leading supplier of braking systems and components, simulators and control systems to the train industry, the company uses real time analytics of data collected from train tracks to define the best driver strategies.

This might range from warning an engineer to back off the throttle five per cent to increase fuel efficiency, or alerting an engineer that gravitational forces threaten to create a dangerous situation a few miles down the track.

4. DevOps: Developer and IT Operational Analytics

In 2015, growing numbers of organisations will be using analytics around DevOps (IT Operational Analytics) to drive software quality and deliver what customers are looking for.  For example, when releasing a new web add-on, mobile app, or feature, companies can analyse the data generated as customers interact with it, to measure performance, identify issues and improve / refine the tool.

As a result software products will get to market faster and be driven by customer feedback and adoption analytics. This process can also drive operational intelligence in other areas that will be fed into overall business decisions.

5. Mobile analytics: Extending the reach of customer service to mobile

According to ComScore, more than 60 per cent of consumers’ time spent online with retailers is on a mobile device. The mobile app is therefore becoming as valuable as the website for omnichannel retailers, and a necessity in building a 360 degree view of the customer.

Ensuring customer experience is as good on mobile applications as it is online will therefore become essential in 2015, and securing transactions will be mission critical. Analytics will play a pivotal role in both, helping retailers tailor the customer journey for mobile and spot unusual event patterns that could suggest potential security threats. DevOps will also be central to ensuring quality of releases and the application delivery lifecycle using mobile Application Performance Monitoring (APM).



Read more: http://www.itproportal.com/2015/01/07/5-reasons-2015-will-year-big-data/#ixzz3QTmMUziE

How Big Data Will Impact the Super Bowl

How Big Data Will Impact the Super Bowl.

B_SuperBowlBigData

It’s almost time for the big game–the one and only, Super Bowl.

As diehard and casual fans alike pick up the snacks and set up the living room for their Super Bowl parties, sports experts are spending hours on end pontificating on the minutiae of the tiniest details happening on and off the field, all in an effort to predict the winner between the Seattle Seahawks and the New England Patriots.

Meanwhile, even those people who admit they don’t care about football will be anxiously awaiting Super Bowl Sunday, if not for the game, then for the commercials. This late January, early February tradition has been around for decades now, but we might just be on the verge of seeing a major disruption in how things are done.

The talk now, at least among certain circles, is slowly moving toward a topic not normally associated with the Super Bowl: big data.

And while most fans may not notice the difference, big data has the potential to change the Super Bowl experience.

One of the favorite pastimes in all sports, not just the NFL, is to predict the outcome of each contest. Most of this is done all in good fun, but some people look at the practice as serious business; and nothing represents the pinnacle of sports achievement quite like the Super Bowl. Predicting a Super Bowl winner is far from an easy task, especially when both teams are so evenly matched, but many experts have turned to big data in the hopes it can provide added insights on game outcomes.

In the average football game, there are numerous statistics that sports analysts have to keep track of, from individual player performances to overall team stats. Big data can go even further, measuring things like total distance traveled, effect of weather conditions on individual plays, and comparisons between different player matchups. It’s essentially taking statistic analysis to the next level, which can, in turn, reveal new details that might otherwise go overlooked by even the most veteran sports pundits.

From all these different stats and figures, big data algorithms can be created to come up with an eventual winner in any game. The challenge to create the most accurate algorithm is one that many businesses and institutions have taken up.

One company, Varick Media Management, created their own Prediction Machine that boasted a 69 percent accuracy rating during the 2013-2014 NFL regular season as well as an impressive record for other championship games. SAP also uses an algorithm based on the NFL’s public statistics database, while Facebook tries to predict a winner from an analysis of social media data. Even though these algorithms take into account a lot of data, the results are far from being 100 percent accurate.

After all, while Varick Media Management accurately predicted the Seahawks would win last year’s Super Bowl, both SAP and Facebook predicted a Denver Broncos victory.The end result was a Broncos’ blowout loss.
Going beyond sports analysis and even the big game, big data may have a big impact on the thing many fans anticipate most: the commercials.

In fact, the commercials may be even more popular than the Super Bowl, itself. According to big data collected through social media listening tools, experts were able to get a picture of what people talked most about before, during, and after the Super Bowl. Based on social media conversations from last year’s championship game, it becomes clear that most people prefer to talk about the advertisements over the actual game.

Interestingly enough, the data also indicates most talks about the Super Bowl happened after the game was over. Based on these findings, experts are saying companies may start rethinking their advertising strategy, viewing online advertising as even more effective than running a Super Bowl commercial. Super Bowl ads cost millions of dollars; and research seems to show that only about 20 percent of those adslead to more products sold.

Instead, the idea is that with big data, companies will be able to reach more customers through their mobile devices, which is more important than ever as businesses and employees look at bring your own device (BYOD) polices and other advantages. Big data, essentially, represents a unique business opportunity that can create more targeted advertising featuring more better engagement, making it a better return on investment than airing during the most watched event on television.

The Super Bowl remains an exciting game that tens of millions of people around the world enjoy, but many aspects of the game are likely to change as we move into the era of big data. Whether it comes in predicting the most likely winner or how advertising is handled, big data could have a significant impact, even if most of it is behind the scenes.

In the meantime, fans can still watch some of the world’s best athletes compete at the highest level.

Shazam compagnon idéal de la pub via la réalité augmentée et les objets connectés

Shazam a de grandes ambitions, et elles dépassent largement le simple domaine de la reconnaissance musicale. Le service cherche à devenir en effet le compagnon « idéal » de certaines publicités et se rapprocher davantage des commerçants, en proposant par exemple des expériences de réalité augmentée.

De la musique à la publicité

Tout le monde ou presque connait Shazam, une petite application pratique permettant de reconnaitre facilement quelle musique est en train d’être écoutée. Si vous êtes par exemple dans un magasin et qu’une chanson sort des haut-parleurs, Shazam vous en donnera le titre, l’artiste, l’album dont elle est extraite et ainsi de suite. Le résultat est toujours accompagné de liens vers iTunes et autres boutiques, ainsi que quelques services de streaming comme Spotify et Rdio. Et le succès serait au rendez-vous puisque Shazam compterait pour 10 % de la musique achetée selon l’entreprise. Toutefois, en l’absence de détails sur la manière dont le chiffre a été calculé, on le prendra avec les pincettes de rigueur.

Mais elle ne compte justement pas s’arrêter là. Elle tient à faire de son service une porte vers des contenus supplémentaires en fonction d’un contexte particulier, essentiellement pour compléter la publicité. Certaines sociétés se sont déjà associées à Shazam et il suffit par exemple de dégainer l’application pendant que la publicité passe à la télévision pour obtenir des informations, à la manière finalement d’un QR-code audio. Il s’agirait donc d’un renforcement de cette activité puisque des essais ont déjà été faits dans ce domaine, notamment la publicité pour La Halle avec Jenifer.

The Next Web a pu interroger Rich Riley, PDG de Shazam, à ce sujet.  Les projets de l’entreprise sont nombreux pour cette année mais concernent avant tout le renforcement du service autour de la publicité. Les développeurs travaillent par exemple sur un « Shazam visuel » permettant de relier l’application à une expérience de vente dans des boutiques physiques, pour obtenir des coupons de réductions ou autres.

Fournir un contenu en fonction du contexte

Même la réalité augmentée est au programme. Au CES de Las Vegas, le PDG a ainsi fait la démonstration d’une publicité pour une Jaguar dans un magazine papier. En scannant la page, Shazam reconnait le contenu et propose automatiquement une expérience 3D à 360°. Il suffit alors de déplacer son téléphone pour observer l’habitacle du véhicule, comme si l’on se trouvait à la place du pilote.

On notera que ce type d’interaction existe déjà, comme Ikea l’a montré avec son catalogue depuis août 2013. La différence ici est que Shazam cherche à fédérer autour de sa plateforme les sociétés qui pourraient être intéressées par ce type d’expérience, en offrant un accès via l’une des applications mobiles les plus utilisées.

Shazam a également des ambitions dans le domaine des objets connectés et des « wearables ». Idéalement, l’application serait assez petite et économe en ressources pour pouvoir être utilisée sur des montres et autres, afin par exemple de pouvoir accéder à des contenus par simple pilotage vocal. Une fonctionnalité que l’on retrouve déjà avec Siri, Google Now et Cortana et il faudra voir comment Shazam compte se démarquer. De même, l’entreprise travaille sur des balises, nommées Shazam-In-Store, capables de fournir du contenu Shazam en fonction de l’endroit où l’utilisateur se tient dans un magasin.

Tout cela suppose évidemment des transferts de données et un stockage d’informations concernant l’utilisateur, même si elles ne sont pas nominales. À la lumière de toutes les attaques sur les deux dernières années, on peut donc se poser la question de savoir comment Shazam compte gérer la sécurité de l’ensemble. Rich Riley n’a cependant pas été prolixe sur le sujet, indiquant simplement que des mesures de protection avaient été prises, et que les données n’avaient pas vocation à transiter vers d’autres entreprises.

Havas Group And Universal Music Group Form Global Music Data Alliance ((Viva)LAS VEGAS, Jan. 5, 2015)

Havas Group And Universal Music Group Form Global Music Data Alliance — LAS VEGAS, Jan. 5, 2015 /PRNewswire/ –.

LAS VEGASJan. 5, 2015 /PRNewswire/ — Havas Group, one of the world’s largest global communications groups, and Universal Music Group (UMG), the world’s leading music company, announced the formation of the Global Music Data Alliance (GMDA), a unique partnership that will enable the billions of data points that UMG and its artists generate through music, ticket and merchandising sales, streaming, social media and airplay to be aggregated and contextually analyzed by Havas’ world-class algorithmic and data scientists.  The result will provide new revenue opportunities for UMG artists and labels by creating powerful marketing and advertising opportunities for brands.

The announcement was made by Lucian Grainge, Chairman and CEO of UMG, and Yannick Bollore, Chairman and CEO of Havas Group, at the 2015 International CES.

Lucian Grainge said, “Our commitment to artist development on a global scale has resulted in the industry’s best track record for identifying and breaking new stars.  But our commitment to artists doesn’t end there.  We want to continue to find new revenue and marketing opportunities for all of our artists around the world by leveraging our industry-leading big data tools and working with forward-thinking companies such as Havas to supercharge our efforts to realize previously untapped revenues from consumer brands and other new business partners.”

Havas Group And Universal Music Group Form Global Music Data Alliance -- LAS VEGAS, Jan. 5, 2015 /PRNewswire/ --

Yannick Bollore said, “Music transmits emotions, cultural symbols, and values like no other form of creative expression. By managing the most successful artists and largest music communities in the world, Universal Music Group is at the forefront of the industry and has already gathered unique consumer insights and databases to empower its labels, artists and fans. This first Global Music Data Alliance will allow our clients and other brands to further expand the common passion they share about music with fans and create more meaningful experiences for them.”

Havas Group And Universal Music Group Form Global Music Data Alliance -- LAS VEGAS, Jan. 5, 2015 /PRNewswire/ --

As part of the initiative, UMG’s proprietary data across multiple artists and genres will be layered with Havas’ behavioral data to allow for a greater understanding of the correlation among artists, music fans and brands. This data includes not only music and video sales and streaming, but also social media and airplay, and even merchandising data from Bravado, UMG’s merchandising division, and ticket sales data from Vivendi Ticketing, which provides ticketing services for select UMG artists and events.  The result is a comprehensive view of music and music related consumption across a range of platforms.

New audience patterns and segments will be developed that can be applied across thousands of artists’ online and mobile properties, thus offering UMG labels, artists and advertisers unprecedented consumer insights which can be used to guide marketing and advertising opportunities for brands and artists alike.  With GMDA, artists will be able to monetize their fan bases more effectively by understanding the different characteristics of their fans and what specific offers and products will appeal to them.

Further, with GMDA, advertisers will be better able to identify which genres and which specific artists appeal to their consumer bases as well as the music-related opportunities that will attract those consumers. This will make the advertiser’s decision to invest in music-related marketing much more accountable and will allow labels and artists the opportunity to create a broader relationship and more integrated partnerships with brands than previously seen.

The launch of GMDA follows a 14-month research program overseen by Havas’ specialist sports and entertainment network, Havas Sports & Entertainment (HS&E), in collaboration with the University of Southern California’s Annenberg Innovation Lab (USC).

The first part of the global research study, entitled FANS.PASSIONS.BRANDS, identified eight logics of engagement resulting in dynamic fan profiles based on a person’s diverse levels of passion and how they interact with football (soccer). Wave two of this multi-methodology study will draw on and evolve these same nuances but with a specific focus on music.

Havas will leverage its group’s research and analysis teams along with some of the industry’s most innovative new start-ups from around the world that specialize in developing technology, to enable UMG and further GMDA partners to derive powerful insights around music and fan engagement.

About Havas Group
Havas is one of the world’s largest and most forward thinking global communications groups. Headquartered in Paris, employing 16,000 people in 120 countries, Havas is committed to being the world’s best company at creating meaningful connections between people and brands through creativity, media and innovation, including data and mobile. To realise this, it is organised to leverage innovation and collaboration between its core teams: Havas Creative Group and Havas Media Group. Havas Creative Group incorporates the Havas Worldwide network (havasworldwide.com), 316 offices in 75 countries, the Arnold micro-network (arn.com), 15 agencies in 12 countries, as well as several leading agencies including BETC. Havas Media Group (havasmediagroup.com) operates in over 100 countries, and incorporates 4 major commercial brands: Havas Media (havasmedia.com), Arena Media (arena-media.com), Forward Media and Havas Sports & Entertainment (havas-se.com). Further information about Havas is available on the company’s website: havas.com

About Universal Music Group
Universal Music Group is the global music leader, with wholly owned operations in 60 territories. Its businesses also include Universal Music Publishing Group, one of the industry’s premier music publishing operations worldwide.
Universal Music Group’s labels include A&M Records, Angel, Astralwerks, Blue Note Records, Capitol Christian Music Group, Capitol Records, Capitol Records Nashville, Caroline, Decca, Def Jam Recordings, Deutsche Grammophon, Disa, Emarcy, EMI Records Nashville, Fonovisa, Geffen Records, Harvest, Interscope Records, Island Records, Machete Music, Manhattan, MCA Nashville, Mercury Nashville, Mercury Records, Motown Records, Polydor Records, Republic Records, Universal Music Latino, Verve Music Group, Virgin Records, Virgin EMI Records, as well as a multitude of record labels owned or distributed by its record company subsidiaries around the world. The Universal Music Group owns the most extensive catalogue of music in the industry, which includes the last 100 years of the world’s most popular artists and their recordings. UMG’s catalogue is marketed through two distinct divisions, Universal Music Enterprises (in the U.S.) and Universal Strategic Marketing (outside the U.S.). Universal Music Group also includes Global Digital Business, its new media and technologies division and Bravado, its merchandising company.
Universal Music Group is a fully owned subsidiary of Vivendi.

About Vivendi Ticketing
Vivendi Ticketing comprises the ticketing businesses See Tickets in the UK and the US, as well as Digitick Group in France. Both businesses specialize in the retail and distribution of tickets for live entertainment, sport and cultural events, in addition to providing operating platforms for venues to run their own ticketing services.
Vivendi Ticketing processes annually over 40 million tickets and counts the Eiffel Tower, the Palace of Versailles, Manchester City Football Club and Glastonbury amongst thousands of other clients.
The business also operates as an internal service provider to other Vivendi businesses notably Universal Music Group.

SOURCE Universal Music Group

Music Business and Big Data: Next Big Sound estimates potential hits (based on data from Spotify, Instagram, … )

To Make A Chart-Topping Song, Think ‘Slightly Unconventional’ | Spotify Insights.

What makes a hit song? People have been chasing that formula since the earliest days of the recorded music industry, and nobody has found it. One company that tries, Next Big Soundestimates its success rate at picking songs that will soon make the Billboard 200 (based on data from Spotify, Instagram, and other sources) at only 20 percent.

Here’s another prediction: Nobody will ever predict, with total accuracy, which songs will reach the pinnacle of the charts. That is not to say it’s impossible to make a song with a good chance of doing well, or to figure out what kinds of songs are more likely to become hits given listening data, the cultural preferences of the time, and/or the instincts of pro hitmakers.

It’s a tricky thing, as demonstrated by new research into the audio attributes of over 25,000 songs on the Billboard 100 from 1958 to 2013. The trick: To be a hit, a song should sound different from anything on the charts, but not so different that it falls off of the cultural radar of the time.

To decide what makes a song conventional or an outlier, Noah Askin (Assistant Professor of Organizational Behavior at INSEAD, in Paris) and Michael Mauskapf (PhD student in Management & Organizations at Northwestern University’s Kellogg School of Management, in Chicago) used audio analysis from The Echo Nest at Spotify to create a new metric called Song Conventionality (methodology below).

It’s ‘Only’ At The Top

Their graph shows that songs in the top 20 show the least amount of conventionality out of any section of the Billboard Hot 100 over time. The farthest outliers, from a musical perspective (based on audio attributes and genre as described below), are the winners:

Differentiation_Chart1

If a song is too weird, it’s unlikely to make the charts at all, of course; songs at the top of the charts are more similar to each other than stuff from obscure genres of limited (if passionate) appeal.

But within the charts, songs at the top are more likely to sound unconventional than songs in the middle. At the bottom of the Hot 100, we see a bit more deviation from the popular musical conventions of the time, but still nowhere near as much as within the top 20.

Are these findings statistically significant? Yes.

“These graphs are just a descriptive representation of the data; when we run our explanatory models, and control for a host of other effects,” responded Mauskapf. “We find that the relationship between conventionality and chart position is statistically significant (e.g., for songs that appear on the charts, higher levels of conventionality tend to hurt their chart position, except for those songs that are exceptionally novel).“

So ironically, in order for large swaths of the population to connect with a song, it has to sound different from the other stuff that’s popular at the same time. We appear to crave convention, but crave something different most of all.

Unconventionality Reigns Among the Hits

Let’s take a closer look at the very top of the chart, where the same effect can be seen, with a larger effect the closer you get to the coveted Number 1 spot:

Differentiation_Chart2

The top song is the least conventional of the top 10. The top 10 are less conventional than the top 20.

If these results are any indication, if an artist and their people wants to put something out that has a good chance of making it to the very top of the charts, they should make something that stands out from the pack by moving in a different musical direction than everyone else’s releases.

So, the moral of the story: Do something different. What, exactly? That’s the hard part.

(As if on cue, as we prepared to post the article you’re reading now, we spotted an article from Slate about how varied the hits were this year, jibing with this research.)

Researchers’ Summary

From Askin and Mauskapf:

  • “When evaluating cultural products, attributes matter, above and beyond social influence dynamics and symbolic classifications like genre.
  • “Attributes shape performance outcomes directly and indirectly, through a relational ecosystem of cultural products we call ‘cultural networks.’
  • “Songs that are slightly less conventional than average tend to outperform their peers on the charts.
  • “Nevertheless, predicting hit songs is nearly impossible to do, because performance is largely contingent on a song’s relationship to other songs that are produced and released contemporaneously.”

Behind The Scenes

“We used The Echo Nest’s attributes to build a ‘song conventionality’ measure and construct networks of songs for each week of the Billboard Hot 100,” explained Askin and Mauskapf in a summary shared with Spotify Insights. “[The below figure] shows one such network, in which the ‘nodes’ are songs and the ‘ties’ between them represent shared genre affiliations and greater-than-average attribute overlap.”

“Our findings suggest that the crowding of attributes within a cultural network can hinder songs’ movement up the charts.”

Here’s a depiction of one song network they made showing their audio and genre similarities (explanation below):

differentiation3

“The spatial relationship [in the chart above] is a function of both a commonly-used network layout algorithm (Fruchterman-Reingold) and of attribute similarity, such that the greater the distance between two songs–>the more dissimilar those songs are across the Echo Nest attribute space (measured using cosine similarities). Colors represent genres; not surprisingly, songs of the same genre tend to cluster together, and certain clusters(e.g., rock and pop) tend to be more sonically similar than others (e.g., rock and funk.soul). Notice however that some songs do not fit the genre clustering pattern, and act instead as brokers between two or more genres (e.g., Little Latin Lupe Lu).”

For any other music scientists who happen to be reading this, here’s some further background on how this research was done.

“1) First, we used a cosine similarity measure to assess the overall degree of Echo Nest audio attribute overlap for each song pair on a particular chart. Put another way, for each song on every chart, we calculated 99 cosine similarity measures to represent the degree of attribute overlap with every other song on that chart. Cosine similarities vary from 0 to 1, and are a common way to measure “distance” across a multi-dimensional attribute space.

“2) The above measure represents songs’ raw attribute similarity, but two songs that have similar sonic attributes may be perceived differently if they are embedded in different genres. Because listeners’ perceptions of a song’s attributes are likely to be influenced by genre affiliation(s), we wanted to weight each song pair’s cosine similarity by the average attribute overlap of those songs’ “home” genres. To do this, we calculated yearly attribute averages for each genre, and then used the same cosine similarity equation to measure the average attribute overlap of each genre pair. The resulting weights were then applied to the raw similarity measures for each song pair. For example: if one rock song and one folk song had a raw cosine similarity of 0.75, and the average cosine similarity between rock and folk is 0.8, then that genre-weighted cosine similarity for those two songs would be 0.75 * 0.8 = 0.6.

“3) After we had calculated genre-weighted cosine similarity measures for each song pair on each chart, we calculated the mean. The resulting value represents each song’s “conventionality” score for a given week. The higher a song’s conventionality score, the more alike that song is to other songs on the chart.

“The average genre-weighted song conventionality score across Hot 100 songs was a little under 0.8, which suggests that, for the most part, songs that achieve some level of popular success are very much alike. In our analysis, we try to tease apart small variations in this measure to explain why, controlling for the effects of genre, artist popularity, and a host of other factors, some songs tend to do better than others.”

You can contact the authors of this research at m-mauskapf@kellogg.northwestern.edu andnoah.askin@insead.edu.

Havas launches the world’s first “Meta DSP”

Affiperf, Havas’ programmatic pure player, extends boundaries of programmatic buying with the introduction of the world’s first real time, agnostic system to work across multiple Demand Side Platforms

Today, Affiperf, Havas’ programmatic pure player, became the first company in the world to offer brands the opportunity to operate seamlessly across multiple demand side platforms with one single point of contact with the launch of its “Affiperf Meta DSP” solution. This represents a significant leap forward in what is now called “the age of programmatic” as the topic continues to dominate the agendas of events such as this week’s Advertising Week in NYC.
As technology, data and algorithmic complexity have increased; automation in the media industry has become the new norm. Despite this, the potential of automated programmatic methods for real-time buying have been limited by the fact that until now, agencies were limited to using inventory from different Demand Side Platforms (known as DSPs) in parallel. As the number of DSPs in the market exploded, this added a rather frustrating and inefficient complexity to the process of optimisation and data collection in programmatic buying.Algorithms data and advertising

Following 3 years of research from Affiperf, a Fields Medal holder and renowned data scientists MFG Labs, the Affiperf Meta DSP solution offers for the first time, a way to unify and make sense of data sets across multiple platforms. It aggregates multiple assets using their APIs, i.e. data inventory, features and algorithms from a number of DSPs. It then uses modelling and decision engines to allow traders to recommend wider, more sophisticated strategic options and monitor them.

Pierre-Louis Lions, MFG Labs co-founder and Fields Medal holder 1994 comments: “Thanks to three years of extensive R & D we have been able to bring technical neutrality to the conception, implementation and optimisation of campaigns. This works both in the real-time bidding process as well as the design for even more integrated approaches that will enable us before the end of the year, to start managing our Affiperf Meta DSP solution for online and offline data and media.”

A unique answer to growing complexity

The Affiperf Meta DSP is powered by enhanced proprietary algorithms that offer clients fluid digitalisation, optimisation and addressability across formats. This ability to collate results and information into one unified marketing statistic marks the end to complexity in this critical area. Although increasing in size, the competitive landscape is not dominated by one DSP, but a fragmented ecosystem of DSP display, mobile and video, rich media DSPs, each of them having different rules, inventories and features. This makes it increasingly difficult for brands to get consistent answers and to see the bigger picture. Technologically agnostic, this is the first solution that is open to all DSPs and all technologies. Through this platform brands can therefore take advantage of the best technology available to reach out to and relate to people with greater speed in a more tailored environment than ever before.

Dominique Delport, Global Managing Director, Havas Media Group and Chairman of Havas

Media Group France and UK comments: “In today’s world, media is code and digital campaigns are like software. The idea behind programmatic when it first started was to secure instant contact between traders and brands that would enable our clients to benefit from an infinite number of connections with consumers in real-time. The explosion of data and the significant rise in the number of DSPs on the market has meant that this promise of programmatic was lost to complexity and silos.

Affiperf Meta DSP disrupts the market with the creation of one single tool that enables our clients to optimise choice across multiple platforms. As a result, our industry can finally take programmatic buying to the next level to help brands generate more tailored, more effective and more meaningful connections with people. This is programmatic without compromise.”

A worldwide roll-out

The initial roll out of the Affiperf Meta DSP includes, amongst others, the recently launched ONE by AOL, onto one open infrastructure. Accessible in over 102 markets, Affiperf will continue to develop the product in the coming months to increase the number of DSP platforms that can be analysed at the same time.

The Meta DSP was launched at the AOL Annual Programmatic Upfront in NYC during the first day of Advertising Week 2014

Algorithmes devins : les smartphones, bientôt nos « deuxièmes cerveaux » ? – Le nouvel Observateur

Algorithmes devins : les smartphones, bientôt nos « deuxièmes cerveaux » ? – Le nouvel Observateur.

Très intéressante interview de Rand Hindi  dans le Nouvel Obeservateur: Patron de Snips, une start-up qui ambitionne d’allier « big data » et algorithmes pour repenser le quotidien et la façon de vivre des citadins en anticipant leurs comportements. Le prestigieux MIT (Massachussetts Institute of Technology) lui a décerné en avril dernier le titre de meilleur start-upeur français de l’année.

Extraits:

Rue89 : Pouvez-vous nous expliquer ce qu’est l’analyse prédictive ?

Rand Hindi : L’idée de l’analyse prédictive, que souvent les gens appellent « big data » par amalgame, c’est de récupérer une grande quantité de données, d’extraire des motifs statistiques à l’intérieur, et de s’en servir pour essayer de prédire ce qui va se passer après.

Mais quand on parle de prédictions, on ne parle pas de prédictions façon « boule de cristal », on parle en fait de mesurer la probabilité qu’un événement particulier se produise.

Comment obtenez-vous ces données qui permettent de bâtir les modèles prédictifs ?

Et par dessus ça, on a également de la donnée qu’on récupère directement à travers l’application sous trois formes différentes :C’est là où ça devient intéressant. Il y a de l’open data, énormément. On capture de la donnée depuis une centaine de sources d’open data, allant de la météo jusqu’à de la donnée de cartographie, sur les gares et les immeubles à proximité… On croise cela avec des données que nous a confiées la SNCF sur l’affluence dans les trains.

  • les requêtes de recherche ;
  • les « check-in » ;
  • et, lorsque l’utilisateur accepte, on récupère sa géolocalisation anonymisée quand il est dans le réseau de transport.

C’est dans cette direction que vous avancez maintenant avec Snips ?

Le nouveau produit qu’on est en train de lancer qui sera prêt d’ici la fin de l’année est une app qui est capable de comprendre ce que tu cherches à faire dans ta journée pour te proposer directement l’information dont tu as besoin et te renvoyer vers les apps que tu as besoin d’utiliser.

Plutôt que d’aller chercher ton application microphone dans ta liste, ton téléphone aura compris que tu es dans un café pour une interview et te propose directement l’appli et une fiche LinkedIn sur moi. Ton téléphone s’adapte à ton mode de vie.

Lire l’intégralité de l’interview: http://rue89.nouvelobs.com/2014/08/24/algorithmes-devins-les-smartphones-bientot-deuxiemes-cerveaux-254326

en savoir plus sur Snips:

At Snips, we leverage personal, social and urban data to make our proprietary machine learning algorithms able to determine the users’ context and predict what they will want to do next. This ground-breaking technology enables us to create products that can learn and adapt to us, taking us a step further towards a world of truly ubiquitous, context-aware computing.