In recent years, the data space in large companies worldwide has been one of the most experimental fail fast and fail forward places to work.
Priorities rapidly shift. There’s always something shiny and new on the horizon.
Data science. Big data. Cloud. Machine learning. MLOps. Digital transformation. Customer 360. Generative AI.
I landed my first analytics job at one of Australia’s ‘Big Four’ banks at the beginning of the firm’s big data journey. At the time, a small team of 20 had just spun up a Hadoop cluster with ‘primitive’ big data tools hosted on difficult-to-scale on-prem infrastructure.
User experience and onboarding were almost nonexistent. You had to be a technical expert to use any of it. (And an expert I was certainly not!)
Things have changed a lot in just five short years. Our Data Platforms team has flourished into a force of 2,500 data professionals, currently feverishly working to build a…
- A suite of compelling internal data products accessible to colleagues via. a Netflix-style data marketplace;
- Next-generation big data platform — built on data mesh principles and hosted on Microsoft Azure cloud;
- Forward-thinking culture, skills, and tools that empower citizen analysts and hardcore data scientists alike through data democratisation.
The urgency is palpable.
Tech-savvy consumers now expect super-personalised digital services. But delivering means harnessing cutting-edge real-time analytics to tackle an insanely vast and diverse data flow. No piece of cake!
Competitors are all racing to go data-driven, aiming for a competitive edge by anticipating customer needs, crafting superior products, and streamlining processes. It’s an intense race.
Plus, mounting regulatory demands for improved data sharing and reporting are…