The use of web scraping has been steadily trending upwards. There’s barely any business that can’t benefit from automated external data acquisition. Ecommerce, however, is one of the few industries where I predict that web scraping will soon become unavoidable.
According to McKinsey, ecommerce margins are often paper thin. Usually, these issues are resolved by scaling the business upwards. While the margins don’t increase by themselves, the overall turnover booms and profit rises in tandem.
There’s an unfortunate drawback to expansion - it’s costly. There might always be more markets to conquer. Getting there, however, takes investment, which again cuts into profits. If a business is savvy (or lucky), these wheels keep turning until they become an ecommerce giant.
Scaling to the edge
In economics, the concept of economies of scale states that expansions and reorganizations lead to overall improvements in productive capacity. As a result, the profitability of a business increases due to better access to resources.
A practical example that applies to retail is the procurement of goods and stock. Smaller businesses will generally get worse deals simply because they cannot afford to buy more goods or resources. As a result, simply expanding the services rendered provides access to better margins.
In fact, in rare cases, reaching extreme scales might cause a business to become a monopsony. Markets with single buyers run into issues, since the business can dictate prices simply because the other side has to sell in order to continue operations.
All economies of scale that do not become monopsonies run into one issue - diminishing returns. There’s only so much to conquer with maximum efficiency. In other words, each new dollar invested in a business might not be as valuable as the previous dollar.
Eventually, investing in expansion and certain arms of business is no longer as effective as it used to be. Of course, as long as the predicted ROI is still positive, it might still look like a good idea to continue investing in the same old strategy. Only, if we don’t take opportunity cost into account.
Reorganizing the ranks
At some point, you have to look elsewhere for better margins. That’s what large ecommerce companies have been doing. They’ve been looking at and analyzing data. Not only the data they own but one that lives in all of the nooks and crannies of the internet.
Most of them outsource external data acquisition to companies that provide such services. Some of them choose DaaS (data-as-a-service), others choose ecommerce scraping solution providers.
From our eCommerce Scraper API data, we can see that there has been growing interest in smaller ecommerce players in the market while the interest in giants has remained steady. As such, we see a turn towards more diverse and universal data.
Data from all sources is combined to produce fascinating results. While there are bread-and-butter data-driven strategies such as dynamic pricing, some giants have been pursuing loftier goals.
Even back in 2014, Amazon had filed a patent for “Anticipatory shipping”, which supposedly uses big data applications to predict buying intentions and allocate products to closer warehouses. Buyers, once they finalize their decision, would get their products faster.
Information, however, isn’t exactly of the same nature of economic expansion. The latter is a little like taking over the center of the board in chess, gaining access to better mobility and the ability to use existing resources better. The former is closer to being able to see the opponent’s hand in poker, which lets you always win in the long-term.
Informational dominance
e business world has been shouting about the idea of being “data-driven” for quite some time now. While it has been met with varying degrees of success, some industries, ecommerce being one of them, have integrated new forms of data quickly and efficiently.
As always, as one business gains an advantage, competitors have to catch up. Nowhere is the pressure more pronounced than in data acquisition and analysis. Business competition is a game (in the sense of mathematical theory), where participants have partial information about possible decisions. They then make them based on the available information.
Let’s take a physical retail store that has no digital front and no digital competitors. Pricing strategies adopted by the retail store will be based on internal data (such as associated costs) and some external data (such as predicted demand). Any competitor, ceteris paribus, would follow much of the same strategy. As such, the game is largely informationally symmetrical.
Once, however, one of the players gains access to information that is not available to others, his advantage increases exponentially. Informational asymmetry means that the one who holds more data can take advantage of limited availability decisions.
An example of a limited availability decision is being the first to enter the market with a specific product (otherwise known as the first mover advantage). Data, especially one acquired through public web scraping lets businesses discover new market needs. These businesses can take advantage of a usually short-lived monopoly in some niche.
Other strategies, also enabled by web scraping, provide a distinct advantage. The much lauded dynamic pricing presents such an opportunity. Being able to better adapt to changing internal and external circumstances means the competition might get priced out of the market.
Competition is a game distinctly driven by information. Any advantage in this area is everlasting and all-encompassing. If one player gains such an advantage, all others must follow suit. Otherwise, they will inevitably be outplayed in the long run.
Since web scraping has already become a driving force in ecommerce, we shouldn’t expect that to change anytime soon. Or even ever. In industries where margins are small and scales large, every small bit of an edge against the competition is likely to be held onto with a stranglehold. As web scraping grants that edge in a unique way, all ecommerce businesses will eventually have to make use of it.
- Here's our list of the best ecommerce hosting services and best payment gateways
