Ellipsis Reports a Change in the Weather of WordPress

In a preview of a tool slated for future release “as a publicly available resource,” today Ellipsis Marketing shared “some headline stats on the WordPress industry” in their newsletter based on their analysis of data they’ve been collecting in 2022:

• Year to date: -10.4% decrease
• Q1 2022: +9.5% increase
• Q2 2022: -18.8% decrease

While it’s not clear what exactly is being measured, these numbers are based on Ellipsis’s FALCON AI for analysis of “tens of thousands of keywords on WordPress and WooCommerce … alongside advertiser information and activity, and information about each URL in the results for each keyword.”

Ellipsis has reported on WordPress and WooCommerce search volume trends in the past — the Weather Report seems to go beyond that with “about 1000 data points for each keyword.”

As we look forward to future weather reports, the possibilities underscore how data-poor the WordPress ecosystem is, at least in terms of what’s publicly available. Open data is not the rule in open source business. Kudos to Ellipsis for being the first WordPress company to innovate like this and share such broadly useful market data.

As the WordPress ecosystem is in search of new and better ways to assess its health — beyond market share — it will be interesting to learn what data Ellipsis is focusing on and how it’s crunching the numbers. Inspired by MozCast tracking the volatility of Google search results, as a barometer for the weather in the WordPress ecosystem, Ellipsis’ live trend tracking might work nicely — along with the personal stories and anecdotes that make the space such a cooperative one.

Update: James Baldacchino, Head of Strategy at Ellipsis, has explained more in a subsequent blog post:

The Weather Report differs in methodology [from Ellipsis’ search volume trend reports]: we’re blending search volume data with advertiser activity, and crucially, this lets us include clicks as well as search volumes. We’re then surfacing the data much more frequently: going forward you’ll be able to access the report “live” with automatic weekly updates.

The other key difference is the output of the Weather Report is a relative figure, aggregated across the whole industry. The score each week is the 4-week rolling average of the blended search and advertiser activity data we’re collecting. This is then presented relative to the previous week’s score.

In Post Status Slack, Ellipsis’ Managing Director, Alex Denning, explained further:

We’re launching a live “weather report” of the WordPress industry. This will be available publicly, and update automatically weekly, in a couple of weeks.

This is different from the search volume trends: we’re blending the 100k+ data points we store for FALCON with advertiser activity (including clicks), to give a single score for the entire industry. Each week’s data point is a 4-week rolling average so we bias away from outliers.

[….]

Our hope is this helps the industry make better decisions. The data itself is neutral, and in future we’ll have year-on-year seasonality.

On the noted year-to-date decrease, Alex observes, “The weather can be bad, whilst the climate is good.” Years’ worth of data will give us a better sense of the climate now — and whether it’s changing or what we expect or might like to think.

The possibilities for future analyses with this weather data over time will be fascinating to explore in the years ahead. My first thought was that being able to track WordPress market share/active installs alongside this “weather” number might indicate some things about how they (potentially) influence each other.

Nev Harris posed a few tests that could be run on Ellipsis’ “weather” data to validate it as a statistical model of the WordPress market:

Correlation Coefficient: How have search volume and advertising data moved in relation to each other. If they are perfectly correlated using both of them does not strengthen the argument. Think heating water to 150 degrees two different ways and then adding them together the result is the same 150 degree water. If they have a strong negative correlation (which is doubtful) they are fighting against each other.

Regression analysis/t-test: We know what happened in the past. We can run a regression analysis based on the last 3, 5, 10 years of data then do a t-test to determine how accurately the The Weather/The Climate would have predicted what ended up happening.

As we look forward to future weather reports, the possibilities underscore how data-poor the WordPress ecosystem is, at least in terms of what’s publicly available. Open data is not the rule in open source business. Kudos to Ellipsis for being the first WordPress company to innovate like this and share such broadly useful market data.