Market research for a Climate Services ObservatoryGo to marco website
The Living Lab concept proposes a number of guidelines (user-centric design, iterative in-the-wild testing, user involvement & panel management, etc…), yet does not offer much in term of practical guidance on how to run an innovation project in real-life. Agile project management methodologies do offer such guidance yet have been critiqued for not providing a focus on the end-user. The Framework for Agile Living Labs (FALL) projects aim to provide practitioners with actionable guidelines on how to run a Living Lab project in an agile way. Agility means being able to fexibly integrate new information in the project’s roadmap, which is exactly the type of situation that a Living Lab project will find itself in, due to its focus on iterative testing with end-users.
A first version of FALL has been described in http://www.timreview.ca/article/1048. Since then, imec’s Application prototyping and Living Lab (APLL) department have refined its application in numerous projects. This has been revised into FALL v2, based on our most recent experiences. The method is particularly focused on practitioners and its aim is to provide actionable insights on how to perform Living Lab projects in an agile way.
Various agile project management methods exist. FALL is based on SCRUM, which takes an approach that is composed of steps that are limited in time (i.e. timeboxed) and focusses on incremental delivery. SCRUM structures communications in a project, making sure that expectations are managed correctly and that the team discusses critical issues. You can read more on SCRUM on https://en.wikipedia.org/wiki/Scrum_(software_development).
The Process Manager understands the method well and guides the team in how to apply the process of FALL, similar to a SCRUM master in the SCRUM methodology.
The Product Owner has the responsibility of keeping the story backlog up to date from the perspective of the end users and the stakeholders. As such, they represent the stakeholders and makes sure that the project meets their needs. The Product Owner preferably has the skills to understand the needs of end-users, rather than a purely technical skill set.
Researchers take the lead in getting input from users at different stages of the project.
Architects create the systems architecture, update and prioritize the backlog in terms of the stories that are not facing towards the user, such as “the server backend should be able to automatically backup the user data that is stored in the database”.
User Experience (UX) designers are responsible for creating designs that represent the GUI (graphical user interface) of the system. These can be wireframes, clickable prototypes, or GUI mock-ups. It is crucial to note that, although the UX designer holds the skillset to build these artifacts, creating them should never be done solely from the perspective of the UX designer. Core to the philosophy of FALL is that the UX designer should work with the feedback that was gathered from the project actors (other team members, representative end users, etc.).
The Developer is responsible for translating the story back- log into functional applications.
User INVOLVEMENT & Panel Management
As users don’t automatically find their way to one of the FALL projects, a User Involvement Coordinator is a crucial role. They put the ‘user’ in ‘user research’, pave the way for users to participate in FALL projects, and make sure participants stay motivated throughout the project.
Users are involved in the project to bring domain oriented knowledge to the team through co-design and usability testing processes. They are guided by the user researcher. It is important that the users involved in FALL projects be as representative as possible of the user group that will eventually use the outcome of the project.
Like users, Stakeholders are also involved in contributing domain-oriented knowledge. They are not necessarily representative of the eventual user population, however, Stakeholders often hold higher-level interest than users and operate from a policy, commercial, or academic point of view.
The innovatrix is a process-structuring innovation management framework. It makes explicit the current knowledge on which the innovation is based and identifies key knowledge deficits in the form of assumptions that need to be tested.
Based on extensive experience with Living Lab-projects, the framework was built on the following eight innovation-relevant criteria:
Customer Segment, Needs, Current Practices, Value Proposition, Solution, Barriers, Value Capture and Key Partners.
At the start of an innovation project, during a kick-off workshop, each of the criteria of the framework is ‘filled’ with relevant input from the innovator. This input is initially gathered through probing questions in which the facilitator of the innovatrix workshop plays an important role.
This input is then awarded one of the following statuses, based on the nature and strength of the input:
– Assumption (the input is assumed)
– Validated (the input is validated by research)
– Invalidated (the input is invalidated by research)
– New insights and unknown (no input is given).
Depending on the assumption status, the input is mapped on different-coloured post-its: yellow (assumptions), green (validated assumption), red (invalidated) and blue (new insights).
In the initial Innovatrix kick-off workshop, innovation-related assumptions are mapped, key uncertainties are identified and focus for future research activities are set. In follow-up sessions, the initial assumptions are, in dialogue with the entrepreneur,– updated by changing its status. This innovation management framework is being developed in a digital tool to better – and systematically – use this approach in Living Lab Projects
The story mapping workshop should be lead by the Product Owner, or a User Researcher (as described above). It assists the Product Owner in collecting data that represents the needs of end-users and stakeholders. A user story workshop has the following phases:
The output of the user story workshop is a list of prioritized user stories that can be added to the project backlog. This will structure the work that needs to be done. For collecting these user stories, agile development tools can be used, like Atlassian’s Jira (https://www.atlassian.com/blog/agile/how-to-manage-a-product-backlog-with-ease), which imec’s APLL department uses.
Other tools similar to Jira (VersionOne, PivotalTracker, Workzone, Targetprocess, etc) allow user stories to be listed in the backlog and to manage the stories that have been committed in each sprint. A sprint is a period of time during which project work is done. These tools make sure that the teamwork can be tracked. This can, for example, be done by indicating for each story that it is “in progress” or “done”.
An information system requires a number of building blocks to play together in order to allow the functionalities of the system to be realized. This is why an architecture is needed, so the development team know what to build and how to connect the various building block together. In larger projects where different teams work on different building blocks, the architecture will offer guidance on who does what.
For example, an architecture could have a graphical presentation, indicating that a system will have a database that will store data and that will be processed by a server. This server would perform machine learning algorithms and would send out data so that end-users can consult that data through an iOS or an Android smartphone app.
Typically, a sprint will be 2 to 3 weeks. To be able to estimate the effort that is needed for a certain sprint, it is important to quantify the amount of work that needs to be completed through planning poker.
Here’s how to do it: planning poker is a process in which the various members of a team indicate how much effort it will take to perform each user story. This results in essential to the team communication on the nature of the work and allows the work to be planned more accurately. https://en.wikipedia.org/wiki/Planning_poker
At the beginning of each sprint, a number of backlog items are selected and are assigned to the members of the team. At the end of the sprint, the idea is that a prototype should be delivered that can be tested.
This produces valuable insights on the purpose of the system and its usability. Do the end-users feel that the system is one which they find valuable? Do they feel that the system is easy to use? User Researchers and User Involvement Coordinators are essential in setting up and collecting the feedback from users. It is essential that Product Owners pay attention to user feedback and make changes to the backlog to reflect new insights.
After the test phase, the iterative process can start again Transfer the user stories to the backlog (step 4). The test phase will have yielded new insights, which will add new needed functionality (and therefore will introduce new stories in the backlog), make planned functionality obsolete, or changed the priority of planned functionality.
If all the objectives of the project have been met or time or budget have run out, the project should break out of the iterative loop and end the project.
The deliverable of the FALL process as a whole can be either a prototype or a product. A prototype is created purely to learn, like when one is doing academic research or when it is important to first find out how a solution would look before going to market. A product is a full-fledged system that can be brought to market. It is important to note that, when the objective is to do academic research, the learnings from building the prototype should be abstracted to contribute to literature in the domain to which the prototype belongs.
To learn more about the Framework for Agile Living Labs, Innovatrix, or to contact imec application prototyping and living labs, contact: email@example.com