How might we reimagine the user experience of engaging with the best, in-depth sustainability data?
The Economist Group / 2024
Overview
Problem
The Economist Group has produced a swathe of leading information on sustainability which can be used to help companies and governments navigate to their sustainability goals. However, it is in a variety of formats (reports, articles, indices and infographics), spread across multiple unconnected sites and would require too many man hours to synthesise and analyse for each specific circumstance.
Solution
The Sustainability Atlas helps business leaders, policy makers & researchers get informed and understand the sustainability landscape, drive strategies* and create reports using trusted information from a custom-trained large language model (LLM).
Final designs under development,
see process & drafts below
My role: project & design lead
I was responsible and accountable for the whole project up to development, at which point I handed over to a delivery lead, and moved into a quality assurance role. I worked with other designers, developers, sales teams, policy researchers, and the client, to conceive of the idea and execute its design and technical development.
An experiment with LLMs
LLMs are perfectly suited to help users digest large amounts of data and make sense of them. This was quickly earmarked as a likely direction for the project but presented 4 key problems:
Key problems with LLMs for this project
Out of the box LLMs can use unreliable information and be susceptible to 'hallucination'; fine for an individual user, but not for a renowned media company providing information to business and regulatory leaders.
The Group was yet to allow any LLM to train off of its data and feared allowing an AI product to be built would mean the data worked its way into the results for the general public.
The Group was yet to provide any AI service to users and saw it as a threat to its very business model of human-created content.
The Economist has a very specific tone of voice that needs to be replicated and is expected to be adhered to by the business, and by users.
Solutions
Building our own custom-trained and domain specific LLM allowed us to create an experience that only drew from specific datasets, using Recursive Criticism and Improvement (RCI) to reduce the risk of hallucination and shielding our data from the views accessible on the usual platforms.
I carried out an immense amount of stakeholder management across the business to bring trust to the process and balance competing pressures.
Research and concept development
Leaning on existing sustainability research (see Value Chain Navigator project) and informal conversations with site users over the previous year, a survey was conducted to understand user preferences around topics and formats in sustainability.
4 ideas from research of varying scale to create a sustainability focused follow up to the VCN
UX development
UX workshop
An internal workshop ran to align teams on the goal of the project, information we want to include, concerns we have and a potential user journey.
Revising user journey
A series of high level wireframes were drafted to present to users for validation and to agree with the client what would be included in the scope.
Journey refinement and resourcing
Wireframes were drafted further and resourcing agreed between the internal and external tech teams in the partnership.
Initial wireframes and prompt experiments
Wireframes were drafted properly on Figma, where initial challenges with the AI experience were uncovered, primarily that, with such a bounded AI, users were likely to need coaching on how to ask questions correctly and in such a way that would generate satisfactory results.
Without this the experience was likely to be sub par, but with it done in the wrong way, would make the experience cumbersome and uninteresting.
4 problems drove the chat experience
1. How do we manage reputational risk of the publisher, by ensuring responses are not overly long and of high enough quality for the majority of use cases?
2. How do we reduce the time needed for coaching users how to use a bounded LLM when people are used to generalised models?
3. How do we get quick insights for our core business target segment, who are bandwidth and time poor, while balancing exploration needs of the policy and researcher targets?
4. How do we reduce long term maintenance costs by reducing API calls?
The solution was to use pre-programmed FAQs that would satisfy 9 out of 10 of the kinds of questions our users would ask and to store these answers in a database, to be called as answers, giving the illusion of it being an AI generated live response.
The FAQs were written up in partnership with the client, policy teams and tech teams, to ensure they would represent as many question themes as possible, while keeping the options as limited as they could be. A few options for how the FAQ themes could be communicated to users were trialled alongside the themes themselves, with policy and business participants.
AI chat and visual design
Below are drafts of how the chat would function, with some FAQs and the ability for users to customise their FAQ to their specific region, industry, company type or problem area.
Refining the responses
A core issue identified with version 1 of the responses was that the answers were seen by the policy team to be too definite, giving too clear and prescriptive an answer. New designs were created to balance the user need for easy to read and clear responses, with the evolving business requirements.
Original response
Expanded response
Revised responses
Globe design & art direction
A secondary part of the experience was the globe exploration of the data. This would primarily serve the policy and researcher users, who we know would be looking to understand the sustainability performance of countries around the world.
I partnered with an internal developer to design through prototyping using WebGL globes as a base for the code and design.
Light vs dark themes to separate two views of the same data
The globe had to handle country and city data of coverage across all indices, as well as scores in a specific index. The solution here was to alter the theme of the experience, to reflect the change of data.
