Living Lab

Methodology
Handbook
 EDITORS:
Katariina Malmberg (European Network of Living Labs)
Ines Vaittinen (European Network of Living Labs)
USER ENGAGEMENT FOR LARGE SCALE PILOTS
IN THE INTERNET OF THINGS COPY EDITORS:
Jonas Breuer (imec)
Dave Carter (Manchester Urban Institute / University of Manchester)
DISCLAIMER
Anna Sthlbrst (Botnia Living Lab / Lule University of Technology)
The information, documentation and figures available in this deliverable
are written by the User Engagement for Large Scale Pilots in the Internet
of Things, U4IoT; projects consortium under EC grant agreement 732078 AUTHORS:
and do not necessarily reflect the views of the European Commission.
Penny Evans (Bristol Living Lab / Knowle West Media Centre)
The European Commission is not liable for any use that may be made of
Dimitri Schuurman (imec.livinglabs / imec)
the information contained herein.
Anna Sthlbrst (Botnia Living Lab / Lule University of Technology)
COPYRIGHT NOTICE
Koen Vervoort (imec.livinglabs / imec)
2017 - 2019 U4IoT Consortium.
LIVING LAB CASE STUDY CONTRIBUTIONS:
ACKNOWLEDGMENT
Clotilde Berghe, Denis-Henri Faguet (Autonomlab)
This deliverable has been written in the context of a Horizon 2020
European research project, which is co-funded by the European Tanguy Coenen (imec.livinglabs)
Commission and the Swiss State Secretariat for Education, Research Luis Navarro Lopez (Guadalinfo)
and Innovation. The opinions expressed and arguments employed do not
engage the supporting parties. Milica Trajkovic (PA4ALL)

GRAPHIC DESIGN:
Dora Matok (European Network of Living Labs)
Co-funded by Co-funded by the
the European Union Swiss Confederation Nathalie Stembert (Stembert Design)
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Setting the Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

What is a Living Lab?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Living Lab methodologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Living Lab Methodology in IoT Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

FormIT methodology in IoT context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

CaseStudies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Introduction to the case studies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Wearables
CASE M-RESIST. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Health and Ageing

CASE CARE(E)RS RALLY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Agrifood
CASE FRACTALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Smart Cities
CASE SMARTLAB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Learnings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Practical Application of Living Lab Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

End-user engagement toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
User selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Quadruple helix involvement - activities and tools involving all stakeholders. . . . . . . . . . . . . . . . . 63
Stakeholder management using the Bristol approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Panel management - putting the life in the Living Lab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Thank You!. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Foreword
Methodology, ecosystem, community insights to the Living Lab methodology
... during its history the term Living Lab specifically in the IoT ___domain the FormIT
has been given several definitions. What methodology. This section focuses on
connects these characterizations is the describing the innovation process based
common understanding of the source on the progress of the innovation; from a
of inspiration for the whole approach: concept to a mature innovation.
involvement of people.
The theoretical background for Living Lab
The aim of this Living Lab Methodology methodologies is then concretized with
Handbook is to introduce some research recent studies conducted in Living Labs.
background as well as serve as a practical The collected case studies presented in the
guidance for researchers and practitioners third chapter serve as practical examples
on Living Lab methodologies, co-creation of different experiments carried out in the
and user engagement. It also aims to inspire topical domains (wearables, health and
the reader with the lessons learned from ageing, agrifood and smart cities), with
thorough research together with real-life four different approaches on how to follow
cases. The handbook is specifically focusing the innovation process and use the varying
on the topical area of Internet of Things, methods and tools throughout the project.
as it is of key importance for societies and
To best serve the people looking for
individuals in todays world. The handbook
guidance on the implementation of
explains how the Living Lab approach can
activities following the Living Lab approach,
greatly support the research and innovation
the fourth chapter links this handbook to
activities in that area.
a recently developed toolkit, comprising
Top experts of Living Lab research of end-user engagement methods and
and practice have contributed to this tools organized according to the phases
handbook by sharing their knowledge on along the innovation process: exploration,
the most recent findings on the topic. The experimentation and evaluation. This section
handbook starts with a brief introduction is completed with useful guidance on user
to the context: what is a Living Lab and selection for innovation activities. Having
the context of Living Lab methodologies, the aim to widen the context for the whole
including the levels of analysis within quadruple helix private and public sector,
Living Lab phenomena: macro, meso and academia and people the best practices
micro. The different phases exploration, and hands-on tools from experienced Living
experimentation and evaluation of the Labs provide concrete advice on the ways to
innovation process in Living Lab context are implement experiments which benefit from
presented in their respective subchapters. the Living Lab approach.
The second chapter reveals background
Chapter 1

Setting
the Context

What is a
Living Lab?
A Living Lab is an orchestrator of open ideas, tools and technologies that will address
innovation processes focusing on co-creation of local challenges. Its a place for innovation
innovations in real-world contexts by involving and exploring new possibilities but where
multiple stakeholders with the objective to reflection and evaluation are built into the
generate sustainable value for all stakeholders working process to make sure the Living Lab
focusing in particular on the end-users. can be flexible and responsive to the changing
Anna Sthlbrst, Botnia Living Lab needs of stakeholders and communities.
Penny Evans, Bristol Living Lab
A Living Lab is a multi-stakeholder
organization set-up to carry out innovation For me a Living Lab is a creative space
projects that follow the principles of open where people with many different interests
and user innovation and focus on real-life and backgrounds can collaborate in new
experimentation. and imaginative ways. Living Labs are also
Dimitri Schuurman, imec.livinglabs creative spaces for sharing technical skills and
technical spaces for sharing creative skills.
A Living Lab is a place where citizens, artists,
This makes Living Labs very special places.
technologists, businesses and public sector
Dave Carter, Manchester Urban Institute,
organisations can come together to co-create
University of Manchester
 Setting the Context 11

Living Labs can be characterized in multiple take place across many different domains, User engagement: this is rooted already in
ways and serve several purposes. They are both typically in health and wellbeing, smart cities the origins of Living Labs, the key to success Living Lab
practice-driven organisations that facilitate and circular economy, culture and creativity, in any activity is to involve the users already
and foster open, collaborative innovation, as
well as real-life environments or arenas, where
energy and mobility. at the beginning of the process.
methodologies
Despite the multiple different implementations, Multi-stakeholder participation: even if Dimitri Schuurman (imec.livinglabs)
both open innovation and user innovation
Living Labs share certain common elements the focus is on users, involving all relevant
processes can be studied and experimented
that are central to the approach: stakeholders is of crucial importance. These
with, and where new solutions are developed. Living Labs are complex multi-stakeholder
include all the quadruple helix actors:
Multi-method approaches: there is no single constellations where a multitude of activities
Living Labs operate as intermediaries among representatives of public and private sector,
Living Lab methodology, but all Living Labs take place. Based on a systematic literature
citizens, research organisations, companies, academia and people.
combine and customize different user- review and on experiences and observations of
cities and regions for joint value co-creation,
centred, co-creation methodologies to best Real-life setting: a very specific characteristic Living Lab practices, Schuurman (2015) made
rapid prototyping or validation to scale up
fit their purpose. of Living Labs is that the activities take a distinction between three different levels of
innovation and businesses. These activities
place in real-life settings to gain a thorough analysis within Living Lab phenomena:

overview of the context. The macro or organizational level, where

the Living Lab is a set of actors and
Co-creation: typically, especially in
stakeholders organized to enable and
technology projects, activities are designed
foster innovation, typically in a certain
as top-down experiments, benefiting from
___domain or area, often also with a territorial
users being involved as factors rather than
link or focus. These organizations tend
actors. There is an increasing recognition
to be Public-Private-People partnerships
that this needs to change so that users
(Leminen, 2013);
become equal contributors and co-creators
rather than subjects of studies. The Living The meso or project level, where Living
Lab approach strives for mutually valued Lab activities take place following a
outcomes that are results of all stakeholders mostly organization-specific methodology
being actively engaged in the process from in order to foster innovation;
the very beginning.
The micro or user activity level, where
the various assets and capabilities of
the Living Lab organization manifest
themselves as separate activities where
users and/or stakeholders are involved.
The Living Lab methodology, with the common
elements and identified innovation process,
thus can be situated at the meso-level, where

As recommended reading, the following booklet gives a good introduction to the

history of living lab research and activities Introducing ENoLL and its living lab
Figure - Common Elements of Living Labs community (www.issuu.com/enoll/docs/enoll-print)
 Setting the Context 13

the projects are structured based on it. As existing, current state of being, the as-is However, to anchor the individual user the current state. This means getting an
presented before, the following principles are or status quo is opposing possible future involvement activities (micro level) with a overview of the current habits and practices of
core within Living Lab methodologies: active states (Alasoini, 2011). This resonates with methodological framework that follows this users you want to target. A specific focus is put
user involvement, real-life experimentation, design thinking, which proposes an iterative design reasoning, Schuurman et al. (2013) on the current problems they still face, taking
multi-stakeholder and multi-method approach, based on analysis and synthesis, proposed that Living Lab projects resembled into account the specific contexts in which
approaches. Besides these main principles, that facilitates experimental learning, and a quasi-experimental approach. This includes these problems occur. This is done by means
another common aspect within Living Lab alternates between divergent thinking and a pre-measurement, an intervention and a of methods and techniques like observation,
methodologies relates to the different stages convergent thinking (Brown, 2008). Action post-measurement, where the intervention is participation and in-depth interviews.
that are followed in an innovation process. research is then used as a method to build equalled to the real-life experiment. Following
After understanding the users and their context,
From the perspective of the innovator, we these methodologies out of concrete cases the above reasoning, we can distinguish three
one engages in the process of discovering
distinguish between the current state and and projects, carried out within the Living Lab main building blocks within Living Lab projects,
latent needs and wants of the users. Here
the future state (Gourville, 2005), where the (DellEra & Landoni, 2014). following the innovation development phases:
sensitizing techniques are used to dig into the
Exploration: getting to know the current users deeper levels of knowledge, uncovering
state and designing possible future states tacit and latent needs and wants. This leads to
the definition of opportunities for improvement
Experimentation: real-life testing of one or
of the users current state. These materialize
more proposed future states
in possible future states that are thought
Evaluation: assessing the impact of the of. This is done by means of brainstorming,
experiment with regards to the current ideation and co-creation techniques. All the
state in order to iterate the future state ideas and options are then materialized into
concrete concepts that can be co-designed.
In the following chapters, the different stages
are represented and the impact of these is In terms of Open Innovation, this phase can
described on the nature of the user activities be labelled as involving mainly exploration
taking place at each stage. processes. Exploration is defined as purposive
inows of knowledge or technology, aimed at
capturing and beneting from external sources
EXPLORATION of knowledge to enhance current technological
Exploration Experimentation Evaluation developments. First, exploration is used to
The first phase within an innovation project, understand the current solutions people use,
following the Living Lab approach, can be the current habits they display and the current
labelled as exploration. In terms of the New context in which people use these solutions and
Product Development (NPD) process, this have developed these habits. Subsequently,
consists of moving from idea towards concept exploration is used to develop and share ideas
or prototype of the solution. In the language of for solutions to these needs, in order to come
entrepreneurs, this is the problem-solution fit to concrete innovation concepts.
stage, as you identify the problem and fit your
This exploration stage also provides you with a
solution as good as possible with the problem.
certain benchmark of the current state. This
The main goal of this stage is to understand
Figure - Phases of Innovation Process is important, as it allows the measurement
 Setting the Context 15

of potential impacts and effects of the When a prototype is stable enough, the market-fit. In the experimentation stage, measurement of the intervention, it should be
experimentation stage in order to measure experimentation can take the form of an actual ideas can be enabled to mature into a tested possible to quantify your value proposition. A
the effects of the innovation. Therefore, field trial. Depending on the possibilities, this prototype or design, which can now be mapped key question at this stage is: what advantages
this stage also can be considered as the testing can be short to longer term, involve a into a target market and user population. The is the future state able to deliver in terms
pre-measurement before the intervention, few to large amounts of users, and can include goal is to launch and implement the innovation of the current state of your envisioned user
which takes place in the experimentation some specific to all aspects of the solution. into these target markets, based on a go-to- population? This also facilitates determining
stage. In terms of techniques, one should focus on market strategy. pricing levels, as this is much easier when it is
unobtrusive techniques to capture the concrete possible to quantify the impact of your solution.
The focus is on understanding the potential
user behaviour with the solution (doing) and
market, which can be done through techniques This stage can also consist of the post-launch
EXPERIMENTATION avoid only relying on what people say.
as market research, user toolkits for activities, where actual adoption and usage

The second stage within an innovation Summarizing, the experimentation stage puts customization or conjoint analysis for defining of the innovation is monitored in order to

development process can be labelled as the designed solution to the test, as much as a concrete offering. This also involves preparing re-design or add new functionalities according

experimentation. In the previous stage a possible in a real-life context, and allows a a coherent marketing communication and to the needs of existing or new market groups.

certain solution or future state materialized decision to be made on whether to head back strategy. By combining the pre- and post-

into a concept, this stage puts it to the test to the exploration stage to iterate your solution,

by developing and experimenting with a or whether to proceed to the evaluation stage.

prototype. Specific for a Living Lab approach

is the real-life setting in which the testing
takes place. The degree in which real-life EVALUATION
can be attained is linked to the maturity of
The third and final stage consists of evaluating
the design. Prototypes can take on many
the innovation. As the exploration stage
forms, from tangible MVPs (Minimum Viable
provided a benchmark regarding the current
Products) to intangible services or experience
state of the end-users, the experimentation
design prototypes, but their main goal is to
stage simulated an envisioned future state by
facilitate testing of the possible future state.
means of an intervention. The evaluation stage
In the experimentation stage, the innovation
enables to generate a post-measurement
itself is presented as a prototype to the users
of the intervention and compare it to the
in the form of a new solution, which potentially
pre-measurement benchmark, illustrating
triggers new habits and new contexts of use.
potential impact and added-value created by
The goal of this intervention is to understand the innovation.
user reactions and attitudes to the proposed
In terms of Open Innovation processes, this
solutions, and to also capture behaviour, which
stage is aimed at exploitation. Exploitation
is made possible by having the testing take
entails purposive outows of knowledge or
place in as-real-life-as-possible contexts.
technology, implying innovation activities to
Depending on the maturity, the interventions
leverage existing technological capabilities
can be labelled as proxy technology For further reading: Schuurman, D. 2015. Bridging the gap between Open and User
outside the boundaries of the organization. Innovation?: exploring the value of Living Labs as a means to structure user contri-
assessments, User Experience testing, or actual
Related to the entrepreneurship literature, bution and manage distributed innovation (Doctoral dissertation, Ghent University),
field trials. available at https://biblio.ugent.be/publication/5931264/file/5931265.pdf
this stage can also be labelled as the product-
Chapter 2

Living Lab
Methodology
in IoT Context

FormIT
methodology in
IoT context
Anna Sthlbrst (Botnia Living Lab)

To support a Living Lab approach in IoT To support the description of this progress, we
innovation projects, it is not only important to use the the FormIT methodology as an example
go through the different phases of exploration, (Sthlbrst, 2008). FormIT has been developed
experimentation and evaluation as mentioned in Botnia Living Lab (BLL) through practical
above. It is also important to make progress experimentation and experiences from applying
in these phases to ensure that the level of it into all digital innovation processes carried
maturity for the IoT innovation increases. While out at BLL. Today, FormIT has been applied in
the former section focused on explaining the more than 100 user engagement processes
different phases in the innovation process spanning from early need-finding to real-world
related to the actions being taken in the tests of market ready innovations.
process, this section focuses on describing the
FormIT is a human-centred approach to develop
innovation process based on the progress of
digital innovations. It aims to facilitate the
the innovation: from a concept to a mature
development of innovative solutions that are
innovation.
 Living Lab Methodology in IoT Context 19

based on a holistic understanding of peoples where the focus and shape of the design The three main phases are; 1) Concept design; they may take. This is done by obtaining a
needs, paying due consideration to issues of becomes clearer, while the attention of the 2) Prototype design, and 3) Innovation design. rich picture of different stakeholders and user
equity, autonomy, and control in relation to evaluation broadens from a focus on concepts In each of these phases, four stages are carried groups, their behaviour, attitudes and values
actual use situations. FormIT is grounded in the and usability aspects to a holistic view on the out: 1) Explore, 2) Co-create, 3) Implement and by using storytelling techniques and open data
theoretical streams of Soft Systems Thinking diffusion of the system. In this process four main 4) Evaluate, which are repeated in iterative collection methods.
(Checkland, 1981; Checkland and Scholes, stages (see figure below) are undertaken within processes. Besides these three phases, one
FormIT has been developed to focus on
1990), Appreciative Inquiry (Cooperrider and each of the three key phases, as outlined in additional phase is included: the planning.
encouraging users to tell rich stories with the
Avital, 2004; Norum, 2001), and NeedFinding sub-chapters below, to move IoT systems from It stands for planning the project as a whole
purpose of identifying their needs, or underlying
(Patnaik and Becker, 1999). ideas that solve societal challenges to solutions and here it is important to gain as much
rationale, relevant in a particular situation
that are diffused to the identified customers or information as possible about the underlying
The FormIT process is typically carried out in (Sthlbrst, 2008; Sthlbrst, 2012). Focusing
user segments (Bergvall-Kreborn, Ihlstrm- circumstances for the project: its aim and
three phases, each phase consisting of four on telling stories instead of answering specific
Eriksson & Sthlbrst, 2016; Sthlbrst & scope; different perspective on the project; and
stages. The process can be seen as a flower questions about needs and requirements has
Holst, 2016; Ziouvelou et al., 2016). constraints and boundaries that need to be
encouraged users to talk about, and discuss,
accepted.
their situation and dreams independent of
Often, the prototype phase is iterated several any technical solution or artefact. Hence, they
times until the prototype is stable enough to could elevate their perspective from what
be implemented on a broad scale in a real- might be technically feasible to what they
world context (Sthlbrst & Bergvall-Kreborn, consider as desirable and meaningful in the
2008; Sthlbrst, 2008). In the following when situation. In these stories, users talk about their
referring to IoT, we refer to the services that needs in relation to particular situations and
are being developed based on IoT data. usually independent of a specific solution or
artefact. In this way, it is possible to find users
underlying rationale related to their needs of a
IoT concept phase possible final solution.

In the first phase, concept design, the focus When the data has been collected, it needs
is on exploring and conceptualising the basic to be analysed and categorised to give a
needs that different stakeholders have in deep understanding of stakeholders needs
relation to the imagined IoT solution, e.g. and values, here, e.g. different personas can
IoT-data based service. These are the needs be developed to give life to the constructed
that motivate them to acquire and use a needs and values. Value mapping techniques
particular solution. Following the language of can also be used. When an understanding of
Soft Systems Methodology, these needs are the stakeholders needs and values is reached,
part of the Weltanschauung (worldview) that the creation of concepts begins. The aim of the
makes the solution meaningful to use, and they co-create step is to develop several conceptual
may vary and take different forms depending ideas that answer to the needs and values that
on stakeholder, context and situation. The have been represented in e.g. the personas.
challenge in the first step, explore, is thus The concepts need to be detailed enough for
to identify the key needs in relation to IoT the users to understand the added value and
Figure - FormIT stages (Sthlbrst & Holst, 2016) innovations, and the different expressions the objective of the IoT concept. Important to
 Living Lab Methodology in IoT Context 21

note in this phase is that the concept should co-design, cultural probes, interviews and that the prototype is detailed enough for the When the creation of the innovation is
not be detailed and focusing on functions observations and of course the results from users to understand and anticipate how the finalised, it should be implemented in a real-
of the IoT system, it should rather describe the evaluation of the concepts in the concept final solution will work. In the early stages of world context where the end-users can
an idea that answers to the elicited needs phase. The challenge in this second phase is the prototype, implementation (manifestation) interact with it in their everyday context and in
and values. To support the creation phase, to separate between needs of the service of the idea can be made in user journey maps, combination with their other systems, activities
co-creative methods such as brainstorming, and needs in the service. Usually an idea of user flows or service blueprints. In the latter and contexts. To support the implementation
body storming, word concept association can the future solution has started to take form, stages, the prototype can be implemented in stage, it is important to sign agreements of
be used. hence the concepts will be further developed mock-ups and finally in being a functioning matters such as responsibilities, usage and
in the form of storyboards, mock-ups and/or prototype that can be tested in controlled privacy. Other issues important to consider is
When several concepts have been co-created,
software prototypes of the innovation. environments such as a lab. In the later stages, the context in which the project should take
the focus shifts again. In the implementation
the evaluation focus is on usability and bug place. These issues are related to the contexts
phase the conceptual IoT idea is put into its real Broadly speaking, there are two types of
testing, while in the earlier stages, the focus in which the IoT service being developed is
context, which in this phase can be in a scenario prototypes; low-fidelity and high-fidelity.
is usually on usefulness of the prototype. To aimed to contribute to. Users response to an
or a user story describing the concept. Finally, Low-fidelity prototypes are concerned with
support this process, methods such as usability innovation can be influenced by how well it
the concept is evaluated with the relevant developing models that capture what the
testing, A/B testing, guerrilla testing, eye merges into their context and their activities.
stakeholders focusing on the attractiveness product will do and how it will behave,
tracking and/or blink testing can be used. Even things that are not directly linked to the
of the concept in relation to the needs, values while the latter is concerned with details of
innovation can influence the users experiences
and KPIs that were identified. This evaluation design, such as screens and menu structures,
of using the product. Hence, to identify and
can be supported by methods such as concept icons and graphics. This can be presented
IoT innovation phase consider aspects in the expected context, and
evaluation, dotmocracy (voting on ideas with in different forms, some want to develop
how these might influence the forthcoming
dots) or thinking hats having a formative a software prototype, while others prefer As the prototype becomes more mature,
evaluation results, becomes important. At this
approach. easier models. Dependent on where in the the innovation phase begins. In this phase,
stage, it is important to define:
development process the project is, the focus the exploration step consists of the input
for this development extends from concepts to from the previous evaluation focusing on the what in the context might have influence
IoT service prototype final design of the service. One way of doing combination of end-users needs of as well on the IoT innovation as well as;
phase this is to keep the designed concept, with key as in the innovation. In addition, it is vital to
what, in the context, the IoT innovation
needs related to it, visible for the users during understand users expected experience from
can influence. This includes privacy issues,
In the prototype phase, the process focuses the data collection activities, so it is possible using the IoT innovation. In the creation step,
movement patterns, feelings, experience.
on exploring opportunities and stakeholders to relate to these during the discussions. This the focus is to design the business model for
needs in the IoT idea concept as well as Key phase is usually iterated several times as the the innovation and fine-tuning the design of
After these underlying circumstances for
Performance Indicators for the IoT service. prototype becomes more and more mature. it. Small changes and adjustments in relation
the real-world implementation have been
In this phase, the known needs, as well as to requirements are quite common, especially
When the data collection no longer generates determined and agreed upon, the issues that
identified values and KPIs, form the basis in relation to service requirements, as the
new insights and findings the focus again shifts need to be discussed become more focused in
for the vision of the IoT prototype that takes system develops and users understanding
to the create step. In the second iteration, character. The aim now is to get information
form in phase two. That is, when using an IoT of structure, content, workflow and interface
the creation of the IoT concept is broadened about issues related to the boundaries for the
based service, what needs are then important deepens. Based on these changes, changes in
to include basic functions, work flows, and evaluation such as:
for the users. This can be expressed in e.g. the design of the innovation also takes place,
interfaces. In this phase, the creation step can
requirements, functions or visions. As in the as well as general development work to finalise Identifying the target group for the
be supported by using methods such as task
first iteration, this is done through a variety of the IoT innovation. innovation and the evaluation.
analysis or card sorting. Here it is important
data-gathering methods, such as focus groups,
 Living Lab Methodology in IoT Context 23

Setting the time-frames of the evaluation, Performing tests of IoT solutions in real world
aiming at identifying critical milestones. contexts, where the test situation cannot
This means, for example, that it is not so easily be supervised and observed, high
appropriate to do a test of a typical demands are put on the design of the test
seasonal service in an inappropriate to create as authentic usage situation as
season, such as a service warning about possible during the period of test. The creation
slippery roads during summer. This might of an authentic usage situation requires
seem obvious, but it has shown to be deep understanding of the users every day
easy to forget, hence causing the project situation as well as their needs relevant in that
to become a bit inactive, waiting for the situation. Hence, users needs are important
right conditions to emerge. In addition, to incorporate in the design of the test to
the test is often one of the last activities increase the probability that users actually use
in a project; thus, any earlier delays in the IoT innovation during the test period. The
the project become obvious. Thus, the creation of actual usage situations also means
timing of the test needs to be considered creating stimuli actions to encourage users to
throughout the project. change their frame of reference to include a
new behaviour, i.e. a new usage situation. This
Discussing if there are any power relations
aspect is central during evaluations in a real-
that are of importance to consider and
life context since users have a natural inertia
how that can have influence on the
to change their behaviour. Due to that, the
evaluation.
truth about users probability to buy, or use,
When this is done, the last evaluation phase the innovation when it is introduced into the
takes place and now the evaluation is focused market, is impossible to gain during a short
on user experience of the finished service. User period of test. Dealing with innovation means
experience goals can be both positive and to deal with uncertainty, hence it is vital to
negative, for example enjoyable or frustrating. remember that it sometimes can take years
They are primarily subjective qualities and for an innovation to have an actual impact on
concern how a system feels to a user. They users behaviour or attitudes. The main thing
differ from more objective usability goals in that when dealing with innovations is to have a
they are concerned with how users experience process supportive of learning from failures,
an interactive service from their perspective, as well as from successes. An additional issue
rather than assessing how useful or productive that can support these actions is to learn from
a system is from its own perspective. non-users.

For further reading, have a look at the following publications:

Sthlbrst, A., & Holst, M. 2017. Reflecting on Actions in Living Lab Research.
Technology Innovation Management Review, 7(2): 27-34.

Bergvall-Kreborn, B., Ihlstrm Eriksson, C., & Sthlbrst, A. 2015. Places and
Spaces within Living Labs. Technology Innovation Management Review, 5(12):
37-47.
Chapter 3

Case
Studies

Introduction to
the case studies
Four case studies from European Network Within this matrix, the linearizers identify their
of Living Labs (ENoLL) members have been innovation processes as linear: following a
selected to give concrete examples on Living structured and pre-determined phase-by-phase
Lab approaches applied in practice. method. They also use standardized and pre-de-
fined sets of tools. Similarly, iterators are using
The four cases each represent differing standardized tools, but rather than following a
approaches towards Living Lab methodologies linear process, they are performing in an itera-
in the context of exploration, experimentation tive fashion: adapting the process based on the
and evaluation, identified according to their input from previous activities. Unlike linearizers
innovation process characteristics (linear and iterators, mass-customizers and tailors do
versus iterative) and usage of tools (standard- not utilize standardized sets of tools. Mass-cus-
ized versus customized). tomizers instead customize these tools, while
still following a linear process from experimen-
Leminen and Westerlund (2016) categorize
tation to evaluation. Tailors, on the other hand,
these four approaches through their visualiza-
use customized tools and follow an iterative,
tion in the format of a matrix using the terms:
non-linear process.
linearizer, iterator, tailor and mass customizer.
 Case Studies 27

The following four case studies represent these 2. Iterator health and ageing, case
four approaches to Living Lab methodologies, Care(e)rs Rally (AutonomLab)
all selected from different domains typical for
3. Mass-customizer agrifood, case FRAC-
Living Lab activities:
TALS (PA4ALL)
1. Linearizer wearables, case m-RESIST 4. Tailor smart cities, case SmartLab
(imec.livinglabs) (Guadalinfo)

Further reading about the matrix on Leminen, S. & Westerlund, M. 2016. Categoriza-
Figure - Matrix on four approaches to innovation process and usage of tools; tion of Innovation Tools in Living Labs. Technology Innovation Management Review,
Note: Adapted from Leminen & Westerlund (2016) 7(1): 1525. www.timreview.ca/article/1046
 Case Studies 29

WEARABLES

CASE M-RESIST
IMEC.LIVINGLABS
 Case Studies 31

Project background
been included throughout our entire design in the business model around the final product. the end user behavior. It is important for
trajectory. It is important because without For example, financial aids or insurances this research to be ongoing, prolonged and
The objective of the m-RESIST project (www. the involvement of end-users you may end up might be able to offer lower fees if the patient longitudinal, and one of the ways to capture
mresist.eu) is to help patients that have with building a product that nobody wants. At is utilizing the technology, or money can be data is through wearables. This helps also in
resistant schizophrenia, which means that the core of what we do is guiding the design reimbursed for certain treatments etc. In the prototyping, as wearables make it possible
they do not react to drugs given for treatment. process through iteratively building a product business model perspective, involving the to capture end user behavior in such a way
Consequently, they must be helped in other that is as desirable as possible, because we public actors is especially important. that prototypes can be improved based on
ways, and one way of helping them is by want it to have as high a potential for uptake the feedback. Wearables can be used for
doing behavioral therapy and trying help them as possible. Private sector example in detecting stress levels, when
through wearable technologies. The wearable We involved the private sector by using using a particular product or application. This
technology used in the m-RESIST project senses commercially available technologies as well as allows for cross-validating feedback collected,
data and sends it to the smartphones of the Involvement of by working together with private technology for example for affective computing, where
patients, which are connected to predictive other stakeholders companies. When working together with the different signals are combined to form insights
models that try to foresee whether they are private sector, it is important to ensure that the - for example, combining biosignals like the
about to have a schizophrenic fit. If the model Academia goals of the business are well aligned with the sweat on your skin, to correlate with what you
sees potential dangers, communications begin goals of the approach of the project. Although were doing on your device at the time.
One of the aspects of importance in involving
with the patients, caregivers and friends and it sounds very simple, this is something that is
researchers lies with design science research.
family, as well as the psychiatrists treating often missed across projects. When working
Design science is about building research while
the patient. The reason for using wearable
you design; the researcher is giving something
with commercially available technologies, on EXPERIMENTATION
technologies as opposed to other types the other hand, it is important to choose the
back so that others can learn from what they
When building our prototypes, we aim to build
of devices is that wearables pose a very infrastructure that is as open as possible, to be
have done. At the core of this discipline is
the prototypes as cheap as possible. The idea
unobtrusive way to capture biometric data. able to work across different types of devices.
behavioral science and the kernel theory, the
is not to engage a lot of effort into building
Using headsets or other devices on a patient The companies offering wearable devices are
behavioral theory that drives the design of the
something that could be the wrong thing
may produce more accurate data but is of no often attempting to lock in their customer to
application. This means, in our case, involving
in the end, but building very cheap and lo-fi
use when nobody wants to wear these devices their own ecosystem, yet it is very important
the psychologists and the psychiatrists
prototypes as quickly as possible. Because if
on a daily basis. A wearable, in this case a when putting something to the market, that
in contributing with their ___domain-specific
they are cheap to make, they are also cheap
wristband, produces better results because is for people to use, that they are able to use
knowledge, giving a lot of value to the project.
to destroy and remake if you find out that you
people are willing to wear their devices for it across many different types of hardware.
are not doing the right thing. Also, testing your
longer periods of time. Compatibility across the devices is something
Public sector prototypes in a real-life setting is important
that is difficult to achieve, and of course
Working together with publicly owned hospitals because if you are testing something that is
privacy issues concerning the access to data
End-user engagement opened the doors to our panel of end users, but
are very important as well. To quote Neelie
happening on the street, you must also test
the hospitals are also often academic hospitals this on the street, and not in a lab.
Kroes (European Commissioner for the Digital
Besides the patients as end-users in our project, so they are to some degree playing the
Agenda in 2010-2014), data is the new oil In the m-RESIST project we also provided the
we are also closely involving their caretakers academic role as well. Furthermore, they are
and those treating them, i.e. psychiatrists and the key to accessing patient records, and the Living Lab methodology and devised a protocol

psychologists. Additionally, a very important importance in involving the public sector often for conducting the workshops, the Living Lab

component in treating patients is the social lies with access to information. In a follow-up
EXPLORATION co-design sessions, in different countries. Due

network of the patient, particularly their project involving public organizations would be to restrictions regarding resources, cultural
For us, the user researcher is at the core of the
friends and family. All these stakeholders have important as they can play an important role or linguistic capabilities, it was not possible
Living Lab process, what they do is capture
 Case Studies 33

to conduct all of the co-design processes

Methodologies used in
of linearizers. Although some standardized
individually, therefore local partners have been tools were used throughout the project (such
guided in running the workshops themselves. project as personas, MVPs) and iterative processes
These sessions included creating scenarios were followed (such as SCRUM), we iterate
The Living Lab methodologies used in the
for the use of the wearables, together with on the consequential phases as a linearizer.
project include:
the technicians involved, creating wireframes SCRUM is an iterative and agile approach
and studying how people are reacting to these Design Thinking to management that can be used in both
wireframes. Similarly, mock-ups were created Interviews development work as well as design work -
to study the reactions of the participants. going through the different sequential stages
User persona
After the mock-ups, the building of prototypes as you would do as a linearizer, although the
How might we / other workshops create
has begun and we are currently testing our process is very iterative in its nature.
first ideas or understand the problem
prototypes with the end users.
Brainstorming / other workshop to create
ideas for solutions
PROJECT OUTCOME - Usability workshop / other workshop to

EVALUATION & try out, test, and improve, validate, the

solution
FOLLOW-UP
Feedback workshop /other workshop to

The project is about learning things, about gather feedback from users

building prototypes - because the main reason Prototyping

for building a prototype is to learn what Minimum Viable Product (MVP)
works and what doesnt. The project is at the
Community Building
moment ongoing, but in the end the outcome
Scrum / Sprint
of the project will add to the knowledge
on wearables and the different aspects to Our process is based on the SCRUM
be taken into account when building such methodology (www.scrumguides.org), every 2
systems. The commercial actors in the project or 3 weeks we are reviewing the backlog and
can also put this data into building a model of launching a Sprint for meeting the elements
schizophrenia, for example a machine learning in the backlog (www.scrumguides.org/scrum-
model, or building their own tools for gathering guide.html#events-sprint). We also work
data or feeding this data online. In the end, with Minimum Viable Products (MVPs); after
many things are given back to the society and delivering an MVP we already begin planning on
the economy through the project. the next MVP together with the stakeholders.

Throughout this project, we have also learnt

to recognize the fast pace of the wearable
technologies, as the pieces or hardware we
Category: linearizer
Further information about the case:
were planning to work with originally had From the four approaches provided, the
Living Lab: imec.livinglabs
already become obsolete by the time that work m-RESIST project fits best in the category
on the project started. Contact: Tanguy Coenen ([email protected])
 Case Studies 35

HEALTH AND AGEING

CASE CARE(E)RS RALLY

AUTONOMLAB
 Case Studies 37

Project background End-user involvement also by the training organizations that

have special contact with them (during
to share their difficulties and good practices.
They are also very important in the co-design
The Care(e)rs Rally is one of the solutions For us as a Living Lab, the very essence is to training time) and have thus a different phase to imagine adaptive solutions to their
arisen from a collective study Career path work on our projects with the users. Therefore, point of view than the employers. In the reality and constraints.
diagnostic phase, home helpers, nursing
for home care professionals, which aims considering the choice of this theme for our
assistants, an occupational therapist
at improving the quality of home services collaborative study, it was obvious for us to
and representatives of the nurses,
delivered to elderly or disabled people. associate all actors of AutonomLab Living Lab EXPLORATION
physiotherapists and physicians were
interested in the question to clarify the whys
The overall objective of the Care(e)rs Rally also involved. Some employers accepted
and wherefores of the issues experienced by The first phase of the project consisted of a
project is to allow people interested in to involve home care professionals during
the caregivers. diagnostic phase to identify areas of tension
these jobs to discover the realities of these the co-design phases and during the
on the career path of these professionals. In
professions through role-playing workshops The project career path for home care rallys workshop.
this phase, the different categories of users
and discussions with experienced professionals professionals consists of different phases: were associated in the following way:
(see an introductory video here: www.vimeo. framing and follow-up of the project through Involvement of 10 home helpers were interviewed and
com/195953219). To reach this solution, the steering committee / diagnostic / reporting
detailed analysis was carried out - during the of results for stakeholders / co-design of
other stakeholders 2 of them observed in a professional

collaborative study - to investigate the home solutions / implementation / evaluation of situation

Academia
workers professional pathway, as they play solutions. The users were involved differently in 20 in collective maintenance (2 separate
a major role in keeping the independence of each phase. The academic institutions (specialized school,
groups),
older people at home. To make these jobs more specialized training organization) were involved
End-users in this context consists of: 5 medical/psychological assistants or
attractive, a state of the art was established because they have different kind of contact
caregivers were met individually,
to determine the actual working conditions of a The final beneficiaries, who were present with home care professionals, have a different
home care professional, and bottlenecks were through a representative of the Collective vision of their difficulties and their needs. An inter-professional group of the home,
identified on the pathway for which a solution of Association on Health (CISS Limousin) Furthermore, they are concerned with the with representatives of the nurses
at all the steering committees of the improvement of jobseekers interest towards committees (regional and departmental),
was to be found. This collaborative experience
study as well as a mobilization of their this sector, as they provide them training. physiotherapists (regional), doctors
was realized with and by stakeholders from the
members during the different stages of
whole regional ecosystem. (regional) and an occupational therapist.
the project and conducting some rallys Public sector This group made it possible to work
One of the solutions experimented was the workshops.
The public sector was represented by on the definition of the profession of
organisation of a rally, because often people Intermediate beneficiaries, i.e. home care the different institutions related to this home help, the tensions and difficulties
who apply for these jobs do not know the reality professionals. It is not always easy to
sector (Departmental council, Health and they perceived or encountered in their
and their difficulties. The employers use a lot get these stakeholders involved, because
social training regional department, State interrelation with these professionals and
of time to find the adapted applicants and the of the tight resources in this sector.
representative). Their involvement allows more widely this sector.
employees risk to spend time in searching for Employers sometimes ask for delegates
taking into account their constraints and A beneficiary group with 4 elderly
a job that is not adapted to their professional to represent their common interests.
the sectors constraints and relying on policy people with disabilities or caregivers
path. Thus, during the first edition of the rally, When it was possible to involve home
care professionals in the process, this levers. to identify the role they perceived from
in one month, 222 people had the opportunity
option was often retained: integrating home help, the qualities they expected
for example to meet employers, to try technical
them into the steering committee and in Private entities from their home help and the difficulties
aids and to have an exchange about these
the different stages of the project through Employers in the sector as well as training they face.
careers with workers in the field.
their employers and sector managers but organizations are widely involved in the process
 Case Studies 39

During the reporting of results for stakeholders public. The rally supports were also presented know if their expectations had been met and adapt the rally to other territories was created.
and co-design of the study, at least five home directly to two assistants at home and two if so, what were the main points they had on Three territories of the region have shown their
assistants as well as representatives of the people looking for work to ensure that the the rally, and a questionnaire of satisfaction interest and one of them has already plans to
nurses, physiotherapists and physicians, as vocabulary and the materials presented to the was handed to them. An assessment meeting set it up.
well as employers, training organizations and public are appropriate. Finally, the prescribers was also organized with the members of the
representatives of the final beneficiaries were (public orientation and employment services, Steering Committee to be able to trace the
present and participated in the co-creation of insertion structures, specialized school) were remarks of the participants Methodologies used in
the solutions. A very precise casting had been also available to accompany the public in the
project
defined to have at least one representative of experimentation of this rally.
the home assistants and one representative PROJECT OUTCOME A wide range of different methodologies were
The rally was based on seven topics (defined
used during the course of the study:
of the beneficiaries in each group. In the
by the partners), that the participants had to & EVALUATION
co-design groups, participants worked in five
validate with the aim to receive their certificate Living Lab methodologies
groups on three life scenarios of three home From the study Career path for home care
Discover the job of Home help. These seven Design thinking
support professionals, based on interviews professionals : five tracks came out of which
topics were: employers, job reality, working Interviews
with field professionals. Field professionals one was tested: the Care(e)rs Rally of the
conditions, beneficiaries, training, other
emphasized the value of finding themselves homecare trades. Visual interviews / collage
professionals in the sector and professional
in the proposed life scenarios and finding out Observation / shadowing
evolution. The results of this action are very encouraging
how their opinions had been listened to.
with a very good participation of the public: Photo Journal / User diary / Guided tour /
The rally was tested for the first time in Empathy prototyping
220 people, and a great mobilization of the
Haute-Vienne during one month in 2016. It
User persona
EXPERIMENTATION reached 220 people, who participated on
partners: 44 partners including 27 mobilized to
propose a total of 42 actions throughout the How might we / other workshop create
average in 4.5 actions among the 42 actions
Among the innovative solutions emerging from month that the rally lasted. first ideas or understand the problem
proposed throughout the month at the rally.
the study, one was a Care(e)rs Rally of the Brainstorming / other workshop to create
Among the 42 workshops, there were some Following the experimentation phase, during
professions of the home help. The aim of this ideas for solutions
in-situ or simulation workshops, exchange the nine-month evaluation phase of the
action was to make the people interested in
with professionals or with beneficiaries to impact of the action, the users opinion is Service design workshop / other workshop
this profession and to be aware of the different
understand the reality of the job, as well as again being requested through an ongoing to develop the solution together
faces of it, to help them to choose whether or
serious game, game or quiz. survey to find out what they have become Usability workshop / other workshop to
not to pursue this career path.
and what improvements they would like to see try out, test, and improve, validate, the
In order to ensure that the participants
This project was co-built with the sectors implemented for a next edition of the rally. solution
expectations were appropriate to the
stakeholders through the organization of three Feedback workshop /other workshop to
objectives of the rally at the beginning, a In terms of impact, the evaluation of the
meetings bringing together actors of guidance, gather feedback from users
period of interpersonal and intrapersonal activities is in progress. According to the
employment, integration, training, employers
reflection was proposed during the launch results received, the participants were very Social media
in the sector Help at home but also their
conference at which participants were satisfied at the end of the rally. Employers Video support/media support by the final
partners and always a representative of the
invited after the presentation of the action, werent totally satisfied because of the low users
Collective of Association on Health.
to indicate their expectations. Finally, at the quality of applications received. Currently the
The co-construction of this project together with closing conference, the participants feedback rate of training as well as penetration in the
all the stakeholders has allowed to construct was surveyed: through interpersonal and employment of participants of the rally is being
a format that is most adapted to the final intrapersonal time again with post-it notes to surveyed. A kit developing the methodology to
 Case Studies 41

The final and intermediate users as well as the of these two sectors to identify innovative Set up workshops to prototype the
other stakeholders were involved at each part solutions. Two innovations were drafted but innovative solutions along with all the
of this project to ensure the creation of well- didnt find any financing to be developed stakeholders concerned by the subject
adapted solutions. Interviews were carried further. In addition, a students challenge
Experimentation of the solution
out to identify the vision of the difficulties in was organised with 17 students from 5
the professional careers of home helpers and different training programmes. They had to Impact and conformity assessment of the

their knowledge about the good practices answer to two challenges: how to improve solution

in their territory. During the first phase, we working conditions with the help of IoT? How
For each step of our methodology we organized
took benefit of an international event the to make these jobs more attractive with IoT?
co-design sessions with many stakeholders in
CIMA, where we invited people to add their Four groups worked on these questions and
order to validate results, topics and thematic
suggestions on post-it notes and to prioritize provided four innovative solutions that were
and try to adapt the process if needed.
the most important difficulties and the tensions presented on a Pecha Kucha evening.
in the professional careers (visual interviews /
To communicate about the rally, we animated
collage).
a Facebook page and our Twitter to share
During the co-creation phase, associating information about the rally. Nine different
representatives of all stakeholders, we created stakeholders of the rally were interviewed
three personas representing home help. This during the closing conference. These interviews
method allows people to know the daily life are available on Vimeo, and a teaser joining
of home help and be more realistic in the these nine interviews was made: https://vimeo.
conception of innovative solutions. During the com/195953219
workshops the participants were invited to
suggest the topics on the rally and to create
a framework for the quiz diffused during the Category: iterator
rally. For that we used some brainstorming
Our method relies on tools that we already
and world caf methods. The website of the
experimented in other collective study. This
rally was tested in advance by home help
method is split in different phases:
professionals and people who were directly
the target of the rally to ensure that it was Definition of a Steering group to choose
adapted. In addition, we can say that this first the subject to explore and follow the
rally was a test to experiment with the concept different steps of the project
of holding such a rally.
Collective study in order to share the

In order to involve the IoT sector, we organized vision of all the stakeholders on the
subject
- in partnership with a digital cluster (Elopsys)
and a digital network (Aliptic) - a workshop on Highlight the most important difficulties
the theme Digital and Care during which we and good practices
presented some IoT experiences in the home Organization of many workshops to
Further information about the case:
help sector. After these presentations we co-design innovative solutions.
Living Lab: Autonomlab
organized a large network between members
Define priorities to develop a project
Contact: Clotilde Berghe / Denis-Henri Faguet ([email protected])
 Case Studies 43

AGRIFOOD

CASE FRACTALS
PA4ALL
 Case Studies 45

Project background
to gather a critical mass of end-users Public sector speed-dating methodology, PA4ALL Living
(farmers and other actors in the Lab introduced an innovative approach in the
PA4ALL is a unit hosted and supported by the
The main purpose of the FRACTALS project value chain, i.e. agronomists), able to establishment of the collaboration framework
BioSense Institute, aiming to introduce the
(www.fractals-fp7.com) was to support its interact with ICT companies (solution the speed dating sessions and facilitated
Open Innovation concept to the socio-economic
start-ups and SMEs across Europe and help developers) and provide feedback on numerous B2B meetings where people from
system of Vojvodina region. Therefore, we
them in better market penetration of their their applications; both the ICT and agri-food industry presented
have involved the Government of Autonomous
innovative ICT for agrifood solutions based on their problems, ideas, and discussed about the
to provide a collaboration framework on Province of Vojvodina due to their strategical
FIWARE technology. The main results of the same topics from different perspectives. Some
which end-users and developers can work decision-making authority. The Government
project included: of the broken myths which were identified
together; brought its network of agricultural extension
include: Farmers dont know how to use ICTs
services (since they are state-owned) and many
43 newly developed applications with to test and validate the applications (reality: ICT companies are first developing
other relevant experts. Moreover, their position
high market potential by innovative developed by SMEs and Web technologies and then searching for problems
brought credibility to the Living Lab, so that
ICT SMEs and startups coming from Entrepreneurs, through the FRACTALS to solve); Farmers dont want to invest money
farmers (who are reluctant in approaching
12 different countries, nurtured with User Community, by providing the ground in ICT (reality: Farmers are reluctant to share
new initiatives) were encouraged to join the
FRACTALS support for open interaction, without pre-defined information).
initiative. On the other hand, the government
roles between developers and end-users;
Technical training plan and material to received valuable feedback on their plans for
ensure correct transfer of knowledge to support beneficiaries in bringing their further development of agricultural and ICT
obtained by tech team in the previous applications closer to the market, by policies, and insights into day-to-day needs
EXPERIMENTATION
projects gaining insights on what the market and obstacles that farmers are faced with.
In order to facilitate testing of developed
really needs.
Established FRACTALS Validation solutions and gathering feedback from
community through mass screening
Private sector
end-users, ICT companies were matched with
process on farmers and other Involvement of Private companies were involved in two main end-users based on their reported needs in
activities:
stakeholders. Validation community
other stakeholders terms of farming practice, type of crops/
gathers 675 lead users that are working animals, services, etc. After the matching,
co-designing of technical solutions
closely with FRACTALS beneficiaries on companies were put in contact with the end
Academia between ICT companies and agricultural
testing and validation of their solution in users and several meetings were organized
PA4ALL invited scientist in co-creation producers
open innovation environment. where testing methodologies were discussed.
process with two aims: first, to bridge the gap
exploration of emerging issues on the During the next phase, the technical solutions
20 sub-projects performed real-life testing between scientist and lab-based research and
market (the process of development of were set on farmers land and they started to
and validation through PA4ALL, a Living day-to-day farming practice in informal and
ideas dedicated to agriculture and food test them in real-life conditions.
Lab which was enhanced geographically natural manner, and to enhance knowledge
security, validated by a considerable
with a pool of users outside Serbia, and and idea sharing between these two groups.
group of people)
with users that are not so tech adept for The second objective was to inspire scientists
not to skew the results. to work on new solutions based on input
EVALUATION
generated by farmers. In this way, the scientific EXPLORATION After the testing period of 1-6 months on
End-user engagement discoveries will have both impact on world-
At the beginning of FRACTALS project, the
average, two questionnaires were distributed,
class research as well as on everyday life and
an anonymous one to the companies and
End-user engagement was highlighted in the professional achievements of farmers. focus was in connecting the ICT community
one to the end-users. The questionnaire
project with following objectives: with end-users (farmers). Inspired by the
 Case Studies 47

for companies had the aim of evaluating Lab environment but also by organizing
the usability of information provided by the events dedicated to demo days and
Methodologies used in Category:
end-user to SMEs in terms of functionality and matchmaking between these two project mass-customizer
future improvements of the provided solutions, industries.
Two main categories of methodologies were The project was identified as mass-customizer
as well as companies satisfaction with the
Nevertheless, this was not the end of our used in the project: in the approach to the innovation process.
provided services, while the questionnaire
collaboration journey. The end-users have Although the linear innovation process was
for end-users had the goal to evaluate the Brainstorming / other workshop to create
been involved in the co-creation process with followed, the attempt was to adopt, modify or
marketability, efficiency, and user satisfaction ideas for solutions
industrial partners, namely ICT companies even invent new effective tools that would have
of the solutions.
(both start-ups as well as already established Service design workshop / other workshop significant impact on the topic.
More than 30 end-users were actively involved and successful ones) that are developing to develop the solution together

in testing and validation of solutions developed solutions for smart agriculture through series The target was to modify and adjust several
in FRACTALS sub-projects. The final solutions of interactive workshops organized all over the methodologies to provide the best outcome.
were co-evaluated together with the end-users, Europe. The main outcome of the project are Therefore, in conducted speed-dating sessions,
half of whom rated the efficiency of their the fruitful collaborations between different brainstorming was encouraged between
solution as extremely efficient. The second stakeholders, establishing numerous joint farmers and geeks on topics related to
half rated their solution as efficient in terms project initiatives that are purpose-oriented needs of agriculture. After identification of the
of time and resource, and only 2 out of the with broad international coverage. needs/challenges, they worked together on the
30 found their solution as neither efficient or technical solution that would bring benefit to
In the course of the H2020 KATANA project, a
inefficient. both sides.
crowdfunding campaign is organized, where
the end-users of the ICT solutions have an

Project outcome opportunity to test, validate and provide their

opinion on business ideas that cross-country
The results of the specific approach of FRACTALS and cross-sectoral teams established during an
project can be summarized as follows: intensive bootcamp weekend.

A broad-based, robust technological End-users will also be involved in activities

capability to invigorate ICT SMEs value of a BioSense demonstration farm, where
creation in agriculture FRACTALS has state-of-the-art technology solutions will be
supported and nourished the creation of presented and used in a novel way, gathering
43 innovative market ready FIWARE-based a broad range of stakeholders from academic
applications which address concrete institutions, ICT providers, commercial
problems and needs in agriculture; companies from other sectors and many others.
Strengthening the systemic dialogue and BioSense and PA4ALL strongly believe that
cooperation between the ICT industry the most important links in advanced ICT for
and the agricultural sector, FRACTALS agriculture are farmers providing their needs
has initiated and established new models as a relevant and solid foundation for further
of communication and collaboration R&D work within commercial companies and Further information about the case:
by involving end-users in the testing/ academic institutions. Living Lab: PA4ALL
validation assignment through a Living
Contact: Milica Trajkovic ([email protected])
 Case Studies 49

SMART CITIES

CASE SMARTLAB
GUADALINFO
 Case Studies 51

Project background
the Guadalinfo Living Lab network. It has Lab vision and the Living Lab network/system. and boosting this bottom-up permeability
required an active approach in detecting needs The main objective is to merge into one model is a must.
Two main conceptions have made the and including a human perspective in the and conception. 2. Definition & awareness. Researching on
Guadalinfo SmartLab appear (see the global technological and scientific tendencies the current knowledge and information on
introductory video at www.youtube.com/ - opening up and challenging exclusion Public sector: Smart Cities, we realized that the concept
watch?v=vHT29gV9_cE). On one hand, the and elitism. The rural gap seems clear, but Since the early beginning it was clear that was unknown in rural areas, but also that
current development of policies and activities we also refer to a Citizens-Smart Tech the main objective was to create a suitable most of current smart strategies were
related to smart cities technology is reaching a Gap. For example, thinking about the most environment where relations and links among excluding rural areas. In this way, dynamic

high degree of maturity. On the other hand, this technologically advanced building: compact policy makers and citizenship flow freely. The activity was designed and executed,
fluorescent lights, solar panels, automatic main driver in creating the ecosystem was where stakeholders (citizens, promoters,
development in the cities is not accompanied
doors and lighting, etc. all monitored by a huge the Living Lab network and its approach. innovation agents ...) were motivated to
by a similar development in rural areas. Thus,
construct the Smart Rural Concept.
two facts are exposed: high tech and economic number of sensors; but is it taking into account It is performing this strategic and political
At first, the action was held in a limited
development among Smart Cities tendencies, the human perspective? Were the bus stops endorsement that enabled the capitalization of
number of Living Labs among significant
and complete absence of knowledge and near the building considered, enabling citizens the project.
rural nodes, aiming to obtain enough
information in rural areas, i.e. Smart GAP. This to avoid the use of private cars? What kinds
data to trace realistic and contextualized
of materials were used in the construction? Private sector:
gap emerges as an opportunity for Guadalinfo, strategies within the whole Guadalinfo
and for Living Labs in general, because of We are considering not only science but citizen Considering the above described environment
Living Lab network.
their adaptability and capability to influence, science. Thus, it was crucial to involve end-users including government (policy makers) and
citizens, it was easy to engage with private The workshops consisted of three main parts:
through direct links, the regional policy makers. since the early beginning.
entities. Taking advantage of the capillarity initial speech, dynamic section (working in
Considering both these aspects, Guadalinfo is and political vehicle, Guadalinfo LL network groups) and conclusions. The main lesson
a perfect instrument to fill the GAP by linking Involvement of other (composed of nearly 800 rural labs) emerged learnt was related to the question on how to
high-tech Smart City productions and rural and reach the end-users: this can be best achieved
citizen needs. In concrete terms, the significant
stakeholders as the perfect instrument to engage with the
through training agents, who will be in charge of
local and regional private entities, encouraging
steps of actions and iterations executed and This actuation is being designed with aiming them to support the working group activities. replicating and contextualizing the workshop.
designed to overcome the new digital gap are at the quadruple helix model, based on the The main lesson learnt is that engaging these
the following: Detecting the gap; definition & new Living Lab innovation model: Universities, entities is easy after a trustful environment EXPERIMENTATION
awareness; strategic definition. Currently the Governments, Companies and Citizens. involving end-users (citizens) is created.
strategic endorsement is a reality and next After consolidating the strategic endorsement,
actions can be seen as an iterative design: Academia: prototyping and testing can be seen as an
Strategy (Regional Policy making) --> Living
In the iteration process of this project, EXPLORATION iterative process:
Labs (Smart Agents) --> Activities design
the bigger efforts were executed on the
Two significant steps were taken at the early
(Stakeholder Engagement). Strategy (Regional Policy making)
Government and Citizens helix. Once the
beginning of the project in terms of user
commitment in these two helixes is mature and Living Labs (Smart Agents)
involvement:
reinforced, efforts in balancing the four-helix
End-user engagement model are required. Therefore, we are currently
1. Detecting the gap. In our citizen innovation Activities Design (Stakeholder
lab conception, the user/citizen is in the engagement).
The necessity for action and the project concept designing the incorporation of the University of
centre of the action. It confers a privileged
itself emerged from the user involvement, by Granada into the case, tracing synergies and As a result of the initial research and awareness
position in detecting both user needs and
listening to people through our main value: complementarities among their spinoff-Fab of the action, as a bottom-up citizen driven
exclusive tendencies to them. Supporting
 Case Studies 53

process, the two main sectors of intervention Government and the eight County Councils):
Methodologies used in
With the support of the Living Lab network,
were defined as: Developing AndalucaSmart: Smart Cities- several workshops were run at the very
Regions under this line of action are included project beginning (workshops for the first ideas /
Citizen-participation working together in all actions aimed at considering that the service definition). These workshops helped
with local administration (municipalities); The following methodologies were used during
citizen is the centre of every process, politics or to identify interested agents, end-users and
the course of the project:
Smart tourism as an economic technology related to Smart Cities. Everything stakeholders. These key actors were engaged
opportunity. starts in the citizens (their needs), in equality Living Lab methodologies in two working groups that defined the areas
conditions, so that they can participate in the of interest, policy requirements and activities to
In this way more hot-spot actions could be Design Thinking
execution, and everything ends in the citizens run under the Living Lab umbrella. Social media
performed through establishing two working (as beneficiary). Observation / Shadowing was used for dissemination.
groups composed of Local Innovation Agents,
The main cities of Andalucia have strategic How might we / other workshop to create
local stakeholders and significant end-users
representing 8-10 Living Labs influencing
plans and actions intended to develop these first ideas or understand the problem
Category: tailor
politics, services and technologies. We cannot Service design workshop / other workshop
areas. The main activities involving end-users,
ignore the rural zones. This Line of Action groups The project was identified as tailor on the
stakeholders and Local Innovation Agents to develop the solution together
together three actions: Training in Smart Cities, approach towards the innovation process;
(Smart Agents) included: Definition of the sector Community Building
Empowering Citizens and Open Smart Lab iterative processes are obvious in our approach,
with problems and opportunities (from the rural
(www.guadalinfo.es/tenemosunplanparati/). Social media but the set of tools are also contextualized to
perspective), mapping of stakeholders and
the area of application together with all the
resources, citizen awareness and engagement. Thus, we can extract the main capitalization More specifically on Living Lab methodologies, components of the working groups. Also the
aspects of the project: design thinking and observation: By setting up a
One lesson from this phase is that the iterator category may apply, as sometimes,
methodological approach works. Training Smart Agent Network to execute concrete (and and depending on the context of applications,
Policy recommendation: through the
ambassadors, local innovation agents and/ permanent revision) strategies, we are adding pre-defined sets of tools are used: hackathons,
strategic line described above, political
or volunteers that engage with the local a Responsible meaning to the Research & personas, game jams, storytelling, etc.
and governmental endorsement is reality
communities to replicate and spread the word Innovation (RRI). The four clusters in RRI are
and budget supporting the activities is
has been successful. considered within the overall process:
ensured.

Policy learning: to be executed as a Diversity and inclusion

PROJECT OUTCOME bottom-up process. Openness and transparency

- EVALUATION AND Scaling out: main activities and research Anticipation and reflection
were scaled out and replicated in similar
FOLLOW-UP Responsiveness and adaptive change
rural areas or villages through our Living

The main outcome of the project is the strategic Lab network.

definition & policy framework. As a bottom-up Scaling up: using the rural Living Lab and
process, data collected together with end-users the Smart Agent, each local initiative
and stakeholders resulted in the definition of is potentially growing from villages to
specific strategic action lines to be included in counties
the global Guadalinfo LL Strategic Plan. One line Further information about the case:

of action was defined, proposed and approved Living Lab: Guadalinfo

by Consortium council members (Regional Contact: Luis Navarro Lopez ([email protected])
Case Study

Learnings

These cases tell four stories of real-life living Continuous contact

lab activities that have all made a difference in
their own region and ___domain. All the presented
to the end-users:
cases vary by their context and approach It could be easy to reflect and expose
towards Living Lab methodologies and the that this project or this Lab approach
innovation process. However, the common appears as a result of expertise thinking of
elements of the Living Lab approach have engineers, politicians or ICT gurus belonging
been relevant for all of them: multi-method to our company but, fortunately, it is not
approach, user engagement, multi-stakeholder the case. It emerged as an opportunity
participation, real-life setting and co-creation. through listening to the territory. Thus, most
important lesson learnt: being in direct and
The highlights brought up by the cases reveal
permanent contact with end-users creates
some essential learnings for all types of
the perfect environment for serendipity and
experiments aiming at involving stakeholders,
opportunities
placing end-users at the center.
Guadalinfo SmartLab
 Case Studies 57

End-user engagement is more like a

Involvement of
A very important component in treating
marathon race rather than 100m sprint. patients is the social entourage of the patient,
Several approaches
It requires persistence, endurance, and different kinds of users: their friends and family. for processes and tools:
flexibility to adjust to end-users needs, to imec m-RESIST
Regarding IoT, we would suggest to have a From the four approaches provided, the
earn and maintain this engagement.
broader look on the community, i.e to engage m-resist project fits best in the category of
PA4ALL FRACTALS When working together with the private
in the co-creation with other profiles different linearizers. Although some standardized
sector, it is important to ensure that the goals
from those considered as targets. Even if tools were used throughout the project (such
For us, the user researcher is at the core of the business are well aligned with the goals
they are not going to be end-consumers of as personas, MVPs) and iterative processes
of the Living Lab process, what they do is of the approach of the project. Although it
the product/service they are going to force were followed (such as SCRUM), we iterate on
capture the end user behavior. It is important sounds very simple, this is something that is
you to have a look out of the box and that the consequential phases as a linearizer.
for this research to be ongoing, prolonged often missed across projects.
is when serendipity and opportunities spark. imec m-RESIST
and longitudinal, and one of the ways to imec m-RESIST
capture data is through wearables. Guadalinfo SmartLab
The project was identified as tailor on the
imec m-RESIST
approach towards the innovation process.
Benefits of a
Involvement of [] Also the iterator category may apply, as

Reaching and real-life setting: sometimes, and depending of the context

all stakeholders: of applications, pre-defined sets of tools are
rewarding the users: Their [governments] position brought
Field professionals emphasized the value
used.
of finding themselves in the proposed life
to PA4ALL Living Lab credibility, so that Guadalinfo SmartLab
The main lesson has been related to how scenarios and finding out how their opinions
to reach the end-users: this can be best farmers (who are reluctantly approaching
had been listened to.
achieved through training agents, who will be new initiatives) were encouraged to join the
Autonomlab Care(e)rs Rally
in charge of replicating and contextualizing initiative.
the workshop. PA4ALL FRACTALS One of the learnings is that you dont always
Guadalinfo SmartLab have to make things too difficult - often
I would try to engage with the research
theoretical solutions or scenarios are too
One of the solutions chosen by the steering bodies such as Universities since the early
complicated to actually work in daily life. The
committee of the study comes from a beginning. At the actual development point
big advantage in working closely together
home-based caregiver who was surprised it is requiring so much effort to involve them
with end-users is that you get an immediate
and amazed to find her solution adopted. in the action. We would try to co-design the
reality check.
The involvement of users allows both the action involving researchers.
imec m-RESIST
identification of solutions adapted to the field Guadalinfo SmartLab
but also allows them to develop a sense of
self-confidence, recognition and prompts the The main lesson learnt is that engaging

questioning of their practices and leads them these [private] entities is easy after a trustful

to become force of proposals.I have some environment involving end-users (citizens) is

ideas you see, its thanks to you! The week created.

after seeing you and talking to you about the Guadalinfo SmartLab
job, it gave me that idea. Ms T.
Autonomlab Care(e)rs Rally
Chapter 4

Practical
Application of
Living Lab Approach
End-user
engagement
toolkit
As presented in the case examples in the that follows the different phases along the
previous chapter, for the engagement of innovation process. These three phases,
end-users in the experiments a vast number exploration, experimentation and evaluation
of different methods and tools exist - the (presented in the introductory chapter), have
challenge is in finding relevant information and been further divided into 3-5 iterations.
selecting appropriate means. A specific toolkit
Although organized in a manner that the
was created by ENoLL in the context of the
phases and iterations could be followed in a
European IoT Large-Scale Pilots programme
step-by-step manner, from beginning until the
to guide the researchers and practitioners
end, the purpose of the entire process is that it
through the innovation processes, with a
is followed in an iterative manner. This means
special focus on user-engagement. This toolkit
that the different phases and iterations in the
is available online (www.u4iot.eu/end-user-
innovation processes are often overlapping,
engagement-toolkit) and it comprises over
repeating, and mixing in order. Throughout
40 different methods and tools found across
the journey the need to jump back and forth
literature and online, put together in a format
between the different phases is to be noticed.
 Practical Application of Living Lab Approach 61

In addition, practitioners may customize the results that requires the participation of often are expressed in terms of complaints or There is one ground rule when recruiting users
the tools according to their specific needs. all the quadruple helix actors. In the following suggestions. In this case complaints are often to be included in user involvement activities and
Although the tools within the toolkit can sub-chapters the Living Lab experts from anchored to a specific product; hence the that is that the involved user should represent
be followed according to the pre-defined three ENoLL Living Labs Botnia Living Lab, innovativeness in these complaints is limited. the actual end-use as well as possible. This is
instructions as described by the tool, the Bristol Living Lab and imec.livinglabs share The launching customer is integrated right from something that needs to be considered when
situation is sometimes calling for a tailored their knowledge in tackling these issues with the development phases to stimulate design users from a specific group of the society are
approach. concrete advice. In addition to the guidance or participate in development activities. The involved.
on user selection introduced in the next reference customers supply their experience of
The four case studies in the previous chapter In order to select people that are suited for
sub-chapter, three practical examples of Living using different applications; hence their ability
each represent a different approach to involvement activities, such as tests, there are
Lab methodologies on the multi-stakeholder to refer to their previous experience becomes
following the process and usage of tools, many factors to consider. Some guidelines
involvement are presented: the Bristol important. The first buyer customer however,
and the toolkit aims to provide a selection of for selecting users to ensure that they are
Approach, Tips&Tricks and Panel Management. plays a more passive role in the development.
possibilities for the users of the toolkit, rather as representative as possible include the
Finally, the lead users should and could be
than a pre-defined pathway to innovation. following:
involved in all stages of the development
However, the tools have been organized to

User
process, although the same customer does
follow four different tracks, each aiming to
not necessarily always represent them (Enkel,
guide the projects in their selection of the

selection
Perez-Freije & Gassmann, 2005).
different tools for their needs:

A. Use cases: Defining use cases and Anna Sthlbrst (Botnia Living Lab)
specifying requirements, as well as validating
them
An important consideration with user
B. Co-creation: of user needs and solutions, involvement is to know who to involve in
specific tools & methodologies for the different innovation stages. The aim of
co-creation including users in the different phases of
innovation process is to reduce the market
C. Prototyping & Testing: First tests risks. Here users, or customers, with different
and Minimum Viable Products (MVPs), qualifications should be included in the
assessments and evaluations, user innovation processes based on their suitability
acceptance to achieve the expected output. Here, the

D. User research: Methodologies for user requesting customers provide ideas for new

research products from the basis of their needs (Enkel,

Perez-Freije & Gassmann, 2005).
When planning the experiments, some specific
questions need to be taken into account. How much a requesting customer can contribute

Common challenges are related to the is often dependent on the companies ability
to capture their ideas and knowledge, which Figure - Customer Involvement in Innovation Developement Porcesses,
selection of the users and the sustainability of
Note: Adapted from Enkel, Perez-Freije & Gassmann (2005)

U4IoT toolkit for end-user engagement tools and methods online: For further reading: Sthlbrst, A., & Bergvall-Kreborn, B. 2013. Voluntary Contributors
www.u4iot.eu/end-user-engagement-toolkit in Open Innovation Processes. In J. S. Z. W. Eriksson Lundstrm, M.; Hrastinski, S.;
Edenius, M.; gerfalk, P.J (Ed.), Managing Open Innovation Technologies. Heidelberg:
Springer.
 Practical Application of Living Lab Approach 63

Strive to maximize the difference between

different categories of users.

Involve users who are flexible and willing

to change and who have a strong social
competence. One single saboteur can
destroy a development project completely.

The participation must be voluntary.

Strive for a balanced distribution by

gender under the circumstances that
the distribution occurs in the user group.
Traditionally it has been shown that male
participants have led to a development
more focused on technical performance,
while female participation has led to a
development more focused on human Photo by Ibolya Feher, courtesy of Knowle West Media Centre

needs.

Quadruple helix
that address local challenges, in a collaborative contribute their knowledge and experience
To maximize the difference among the
process of innovation, testing and exploring by facilitating introductions between different
use categories, all kind of ages need to be
new possibilities. Tools and practices are stakeholders and offering training and advice

involvement -
represented.
co-designed through exchange and dialogue for stakeholders with little previous experience
Focus in the selection should be on the with local people, national and international of working within communities.
users who are the least knowledgeable
about the area.
activities and networks, organisations, academics
individuals with expertise in relevant fields.
and
Knowle West Media Centre (KWMC) has

tools involving all

developed The Bristol Approach to Citizen
Do not necessarily reward the users for Bristol Living Lab uses a range of creative
Sensing, which brings together Quadruple
their involvement. Focus more on finding approaches and digital tools (including IoT

stakeholders
Helix stakeholders.
a good combination of the four Fs that devices) to gather data to support people to

motivates participation: Fun, Fame, tackle issues that are important to them: from Communities and their needs are central to
Fortune and Fulfilment. Penny Evans (Bristol Living Lab) damp homes and data sovereignty to social Knowle West Media Centres work as Bristol
isolation and poor air quality. Living Lab. The organisation has 20 years
Botnia Living Lab defines the Quadruple Helix of experience of working in Knowle West
An enabler-driven Living Lab facilitates
approach as a way of working that means the (a neighbourhood of approximately 20,000
strategy development through action, where
inclusion of representatives from public sector, people in Bristol, UK that features high in the
information is collected and used so a range of
universities, companies and citizens in the governments deprivation indices), and has
stakeholders can co-create knowledge through
innovation process. developed strong relationships and built up
processes that create parity. Through its
trust with individuals, groups and organisations.
Knowle West Media Centre, home of Bristol established networks, a Living Lab like Bristol
The BACS programme puts people and their
Living Lab, defines a Living Lab as a place Living Lab can act as a broker and connector
skills, priorities and know-how at the centre of
where these stakeholders can come together between the members of the Quadruple
innovation, using a framework that Living Labs
to co-design tools, practices and technologies Helix, ensuring that each stakeholder can
 Practical Application of Living Lab Approach 65

and other stakeholders of the Quadruple Helix are able to become more confident users and Work is integrated with other work in From community activists;
can also employ to guide their practice and producers of smart technology rather than cities
For academics working with community
innovation to ensure it not only meets their consumers subject to a technology-led smart Hardware and technology are used as activists;
needs but also addresses wider social concerns city, while public sector, academic and business and when they are useful
For Living Labs working with citizens.
including social inequality, lack of inclusion and stakeholders gain a greater understanding of Citizens roles are central
The advice ranges from being dogged but
diversity in smart city development. community issues and priorities. Projects are open and shared
not inflexible to remembering that experts
Working with public sector stakeholder In 2015-2016 KWMC used the six steps of The Opportunities for new business models should be on tap, not on top. The Tips & Tricks
and enterprises are created resource pack can be used in a variety of ways:
Bristol City Council and the Barcelona-based Bristol Approach to support citizens of East
innovation company Ideas for Change, KWMC Bristol to identify issues that were important from being a useful discussion-starter to gauge
The 2015 report Rethinking Smart Cities
has developed The Bristol Approach to Citizen within their community and which technology differences of opinion and interpretation
From The Ground Up (T. Saunders & P. Baeck, within a team; to raising awareness of the
Sensing: a six-step framework for delivering could be used to tackle. Following a period of
2015, Nesta) identified the absence of these opportunities and challenges presented by
technology and innovation projects that use engagement, KWMC discovered that damp
features as something that could hold smart different cultures, disciplines, languages and
IoT devices, sensors and smart technology to and mould in homes was a significant problem
cities back from delivering real value. The expectations.
ensure that they place communities and their for many people. Through a programme of
Bristol Approach, through further iterations,
priorities at the heart of innovation. Rather practical workshops, hack days, making
promises a process that has smarter citizens
than pushing technology or pre-determined sessions and regular meetings, KWMC

Stakeholder
at its heart and which will help to decrease and
solutions onto people, The Bristol Approach supported people from different backgrounds
diminish existing patterns of digital and social
focuses on supporting people to work together to come together to identify key actions and

management
exclusion.
to pull-in the knowledge, technology and develop and test a damp-busting system
resources needed to tackle a problem. which included: frog-shaped temperature and

BACS is a process where citizens build, use,

humidity sensors, digital interfaces to make
sense of the gathered data, mapping tools
Bristol Tips & Tricks using the Bristol
or act as, sensors, for example identifying initiative: supporting
and gathering information (or data) that will
to visualise the scale of the problem, and
training for volunteers to tackle the problem stakeholders in reflective approach
help them to tackle an issue thats important
practice
on a practical level. The collaborative testing Penny Evans (Bristol Living Lab)
to them. This sensing process could involve
of the Approach in this Dampbusters pilot
creating a bespoke temperature sensor
brought together universities, businesses, In order to support individual stakeholders to
from scratch or using a piece of technology A key factor in successful stakeholder
technologists and open data specialists, city explore different ways of working, learning
that already has an in-built sensor, like a management is the identification of relevant
council representatives, artists, architects, and collaborating with others, KWMC has
smartphone. community and city challenges. By tailoring
housing associations and citizens. transformed many of the principles, practices the content, scope and focus of any project
At the heart of BACS is the development of and lessons it has developed over 20 years into or programme to the interests and priorities
The Bristol Approach offers a process, set of
a city commons, where resources, tools, a series of learning resources called Tips & of the people and partners working with it,
resources and way of thinking about data and
expertise and technologies are shared and Tricks which were compiled after consultation projects are thus defined through co-design
its role, where:
used for the common good. A key commons and collaboration with female activists from and co-production and have an inclusive
principle is that of the low floor/high ceiling, Human behaviour is taken more seriously South Bristol, a group of academics, and Living approach.
which ensures there are no barriers to taking than technology Labs from across the ENoLL network. There are
part (a low floor) but that every stakeholder There is plenty of use generation and three sets of 20 recommendations covering The approach used by Bristol Living Lab ensures

can be challenged to the best of their abilities sharing of evidence Tips and Tricks: the inclusion of individuals and groups at risk

(a high ceiling). Through the framework, people There is a focus on developing data skills of social exclusion and, consequently, digital
 Practical Application of Living Lab Approach 67

and technological exclusion. This approach

works to address the inequality of access to
new technologies and means of production,
such as AI, robotics and digital fabrication,
thereby enabling active citizenship, citizen
participation and equality of access to city
opportunities. This approach ensures that
no technology or pre-determined solution
is imposed on people and that stakeholder-
citizens are supported to collaborate to create
the change they want to see and the tools that
will help them achieve it.

The practice of identifying challenges should

be able to be translated to anywhere on a
global basis and is likely to both uncover
Photo courtesy of Knowle West Media Centre
known common urban and regional issues and
reveal concerns that are less highly profiled
needed or could be collected for the project. 4. If your project involves designing a and inclusivity where everyone is valued for
but equally important. For example, during
Look to understand the challenge from a prototype or product, design iteratively. their knowledge and expertise whether
the Women and Data Futures project at
range of different perspectives and begin It is important to test a basic working that knowledge is technical or not.
Bristol Living Lab, it was discovered that young
to investigate who will benefit from solving prototype of a whole system, rather than
mothers felt that their lack of privacy online 6. Create an engagement team who can
the problem or concern, and who already perfect each piece in isolation. Ensure that
was a big issue, including how and where support stakeholders to test the prototype
is aware and has previous knowledge any workshops or co-design sessions take
their personal information was being shared and learn new skills to help them apply
and expertise. Consider if there are any place in an accessible neighbourhood and
without their knowledge and the value and the technologies and solutions. Encourage
other stakeholders who should be invited venue so that travel is not a barrier to
ownership of this information. openness and share your findings with
to participate. Finally, seek to establish attendance for stakeholders.
other communities facing the same
The following section outlines key learnings partnerships with those who have networks
5. Take steps to address concerns that may challenges so they can utilise the tools to
from The Bristol Approach: and communication channels that can
arise about the user experience: dont effect change.
be utilised to strengthen and support the
privilege or prioritise technology and be
1. It is important to be patient and open- 7. Finally, celebrate achievements together:
work. These collaborations can lead to the
sure to remember to make engagement
minded, as identifying challenges that marking achievements, acknowledging
successful broadening of networks.
with technologies personal, fun and
could be tackled within a community the contributions that have been made,
3. Develop a programme that offers a range engaging. By demystifying terms like data
takes time and requires open, two-way and saying thanks is essential to maintain
of activities that can involve all partners and taking time to explain how different
communication. An open and transparent interest and commitment.
- members of the Quadruple Helix and technologies work during workshops helps
selection process for choosing the
beyond. These other members could include to create an environment of transparency
challenge is essential.
artists, creative technologists, academics,
2. When challenges have been identified, it is
social enterprises and public sector
More details about The Bristol Approach can be found at
essential to uncover what is already known
organisations. www.kwmc.org.uk/bristolapproach.
and what knowledge or information is
More information on Tips & Tricks at: https://shop.kwmc.org.uk
 Practical Application of Living Lab Approach 69

As stated above, working collaboratively is key Most Living Lab initiatives have a clear view on
Panel circles
of new (sub)groups that were not identified
to the success of projects. This collaborative what they want to achieve and create beautiful before.
work should be structured by a clear set of projects & business plans to make sure that Organizing at least one workshop with the
By mapping all the (new) identified (sub)groups
aims and objectives, an understanding of the all involved stakeholders can profit from the Living Lab team and/or different stakeholders
into a panel circle, the relationships between
distinctive assets and expertise that each results. Nevertheless, many initiatives forget to within the initiative offers the opportunity to
these groups will become clearer and more
partner brings, and space for review. Iteration think - in advance - about which users they broaden the views of the (sub)groups of users
structured/clustered.
and reflective learning. This needs to ensure want to involve and how theyre going to deal who are involved in relation to the problem
that technological developments dont act with the management of that panel. definition(s) the Living Lab is tackling. Naturally the (sub)groups in the middle of
to increase and amplify existing patterns of the circle will be more actively involved in the
Whats the role of the panel within the Brainstorming about the problem and the
social and digital exclusion but moreover allow project as the ones out of the centre of the
Living Lab? Which activities do they need to involved stakeholders will widen the view on the
people to gain skills and contribute knowledge. panel circle.
perform? What are the parameters of the panel and will likely lead to the identification
Bristol Living Labs evaluation approach is panel (quantitative vs. qualitative, diversity,
aligned with action research, so it nurtures timeframes)? Who do we need? Where do
reflective practice for all stakeholders. The we find them? How are we going to support
main aim of effective stakeholder engagement & protect them (recruitment, privacy, helpdesk,
is to create an environment where technology, rewarding)?
knowledge, expertise and collaboration can
Furthermore, it is important to involve all
be combined to co-create solutions and
possible users in a Living Lab panel and
opportunities that generate value and legacy
therefore to take a wider approach on who
for all.
needs to be involved. For example, if you want
to involve lonely seniors to solve the problem
of their isolation, its essential that you involve

Panel their caregivers and family or even the general

practitioners as well.

management - Involving users in Open Innovation and

putting the life in

Living Lab projects requires specific skills
and consists of different aspects. Foremost

the Living Lab

panel management facilitates the interaction
between end-users, researchers, instigators/
clients and addresses the tension between the
Koen Vervoort (imec.livinglabs)
different expectations of these stakeholders.

Within a multiple helix Living Lab environment, Over the last years imec.livinglabs developed
user involvement is an important key to a two-step method to keep a project-
success, although way too often this aspect is overview of the defined feedback steps of
forgotten by the different stakeholders in the the panel (segments): first to map all involved
Living Lab. stakeholders in a Living Lab panel and then to
organize that panel in a panel matrix.
Figure - Panel Circles
 Practical Application of Living Lab Approach 71

Panel matrix
users, which research methods will be used
and what material and panel resources are
Once all (sub)groups are identified and clustered available in order to be able to build up
into the panel circle, these (sub)groups can recruitment campaigns, support structures and
easily be transferred to a panel matrix. reward strategies. Naturally this is a working
document which can be updated based on the
This matrix combines the groups with the
changes or findings in the project/initiative.
defined research/user activities of the project/
initiative. It offers the possibility to create an Finally making this matrix WORTHCASE (who/
overview on which groups would have to be what/when, organization, recruitment, timing,
involved in which activities. help & support, communication, attrition,
succeed & estimation) will increase the
By matching them with an X, all stakeholders
possibilities to adjust strategies if necessary
get better insights on what is expected from
without losing the initial planning from sight.

Projectphase 1 Projectphase 2 Projectphase 3

Co-creation session
Intake survey

Field trial
Activity 2

Activity 2

Activity 2
Activity 1

Activity 1

Activity 1

Seniors 65+
Subgroup 1
Subgroup 2
Health care provider
Subgroup 1
Subgroup 2
Carers
Volunteers
Family members
Pharmacists
General Practicioners
Specialist
Hospital employees
Members city government
CPAS-employees
GP- circuit members

Timeline Project X

If you want more information about this method or the WORTHCASE matrix, please
contact Koen Vervoort from imec.livinglabs ([email protected]).
Figure - Panel Matrix
Conclusions
The Living Lab, be it defined as a methodology, The four Living Lab case studies, all with
ecosystem or community, is foremost about their different approaches to innovation
bringing people together to innovate. It is process and selection of tools as well as
the practitioners driving the experiments, operational domains, expose the variety of
people giving their valuable contributions as Living Lab activities. However, as presented
end-users, public administrators, company in the learnings from the case studies, they
representatives or academic researchers. all share the common elements of Living Labs
and emphasize foremost the human-driven
Throughout this handbook, the different aspects
perspective.
have been highlighted that make the Living Lab
approach successful in tackling the innovation Finally, for the guidance of the practical
challenges. The first chapter introduces the application of the Living Lab approach, a
context with key Living Lab characteristics that specific end-user engagement toolkit has
are typical for this kind of innovation activities: been developed to support the selection of an
multi-method approach, user engagement, appropriate methodology or tool (available at
multi-stakeholder participation, real-life setting www.u4iot.eu/end-user-engagement-toolkit).
and co-creation. Three main elements have Necessary considerations in terms of user
been distinguished within Living Lab projects, selection are to be taken into account when
following the innovation development stages: planning the experiments. In addition to the
end-user engagement, the whole quadruple
Exploration: getting to know the current
helix approach, i.e. inclusion of representatives
state and designing possible future states
from public sector, universities, companies
Experimentation: real-life testing of one or and citizens/end-users is to be stressed in
more proposed future states the innovation process. Practical examples
of concrete tools as well as lessons learned
Evaluation: assessing the impact of the from the user selection and multi-stakeholder
experiment with regards to the current engagement specifically in IoT context are
state in order to iterate the future state presented by three ENoLL Living Labs: Botnia
Living Lab, Bristol Living Lab and imec.
The IoT context brings in certain conditions that
livinglabs. Further examples are also available
need to be taken into account with the Living
through the European Network of Living Labs
Lab activities and methodologies. The FormIT
website (www.openlivinglabs.eu).
methodology considers these challenges with
presenting the innovation process with three
main phases to move IoT systems from ideas
that solve societal challenges to solutions that
are diffused to the identified customers or user
segments.
 References 75

References
Gourville, J. T. 2005. The curse of innovation: Technology Innovation Management Review,
Why innovative new products fail. MSI 3(11), 28.
Report, 5(117), 3-23.
SmartIES (Nordic cross-border Living Lab
Leminen, S., & Westerlund, M. 2016. project) 2012. The Living Lab Methodology
Categorization of Innovation Tools in Living Handbook. Available at https://www.ltu.se/
Labs. Technology Innovation Management centres/cdt/Resultat/2.59039/Metoder-och-
Review, 7(1): 1525. www.timreview.ca/ handbocker/Living-Labs-1.101555?l=en
Alasoini, T. 2011. Workplace Development article/1046
Sthlbrst, A., & Holst, M. 2016. Botnia Living
as Part of Broad-based Innovation Policy:
Leminen, S. 2013. Coordination and Lab Handbook, available at https://www.ltu.
Exploiting and Exploring Three Types of
participation in living lab networks. se/research/subjects/information-systems/
Knowledge. Nordic Journal of Working Life
Technology Innovation Management Review, Botnia-Living-Lab/Handbocker?l=en
Studies, 1, 2343.
3(11).
Sthlbrst, A., & Bergvall-Kreborn, B. 2008.
Checkland, P. B. 1981. Systems Thinking, Systems
Norum, K. E. 2001. Appreciative Design. FormIT An Approach to User Involvement.
Practice. Chichester: John Wiley & Sons.
Systems Research and Behavioural Science, In J. Schumacher, & V.-P. Niitamo (Eds.),
DellEra, C., & Landoni, P. 2014. Living Lab: A 18. European Living Labs - A new approach for
methodology between user centred design and human centric regional innovation: 63-76.
Patnaik, D., & Becker, R. 1999. Needfinding:
participatory design. Creativity and Innovation Berlin: Wissenschaftlicher Verlag
The Why and How of Uncovering Peoples
Management, 23(2), 137-154.
Needs. Design Management Journal, 10(2): Sthlbrst, A. 2008. Forming Future IT - The
Checkland, P. B., and Scholes, J,. 1999. Soft 35-43. Living Lab Way of User Involvement. Lule
Systems Methodology in Action; A 30-year University of Technology, Lule.
Saunders, T. and Baeck, P. 2015. Rethinking
Retrospective. New York John Wiley & Sons.
smart cities from the ground up. Nesta, Sthlbrst, A. 2012. A Set of Key-Principles to
Cooperrider, D., & Avital, M. 2004. Introduction: public reports. Assess the Impact of Living Labs. International
Advances in Appreciative Inquiry - Constructive Journal of Product Development, 17(1-2):
Schuurman, D. 2015. Bridging the gap between
Discourse and Human Organisation. In D. 60-75.
Open and User Innovation?: exploring the
Cooperrider, & M. Avital (Eds.), Constructive
value of Living Labs as a means to structure Ziouvelou, X., Alexandrou, P., Angelopoulos,
Discourse and Human Organisation. Oxford:
user contribution and manage distributed M. P., Evangelatos, O., Fernandes, J., Loumis,
Elsevier.
innovation (Doctoral dissertation, Ghent N., McGroarty, F., Nikoletseas, S., Rankov,
Enkel, E., Perez-Freije, J., & Gassmann, O. 2005. University). S., Raptis, T., Sthlbrst, A., & Ziegler, Z.
Minimizing Market Risks Through Customer 2016. Crowd-driven IoT/IoE ecosystems:
Schuurman, D., De Marez, L., & Ballon, P. 2013.
Integration in New Product Development: a multidimensional approach, Beyond the
Open innovation processes in living lab
Learning from Bad Practice. Creativity and Internet of Things: Everything Interconnected.
innovation systems: insights from the LeYLab.
Innovation Management, 14(4): 425-437.
Thank You!
This handbook was initiated under the
coordination and support action User
Engagement for Large Scale Pilots in the
Internet of Things (U4IoT) that provides online
and offline toolkits, workshops and other forms
of support to actively engage end-users and
citizens in the pilot projects of European IoT
Large-scale pilots programme. This project
has received funding from the European
Unions Horizon 2020 research and innovation
programme and the Swiss State Secretariat for
Education, Research and Innovation.

The handbook has been a result of a

collaborative effort of several individuals in
the European Network of Living Labs (ENoLL)
and its members. Special thanks to the main
contributors of the book, namely: Anna
Sthlbrst (Botnia Living Lab), Koen Vervoort
and Dimitri Schuurman (imec.livinglabs) and
Penny Evans (Bristol Living Lab) as well as
Dave Carter (Manchester Urban Institute/
University of Manchester - ENoLL) and Jonas
Breuer (imec) for thorough review and advise.
In addition, the handbook has benefited greatly
of the insightful case studies provided by four
ENoLL Living Lab members: Tanguy Coenen
(imec.livinglabs), Denis-Henri Faguet and
Clotilde Berghe (Autonomlab), Milica Trajkovic
(PA4ALL) and Luis Navarro Lopez (Guadalinfo).
And finally, big thanks to the two graphic
designers, Dora Matok (ENoLL) and Nathalie
Stembert (Stembert Design) for making this
handbook a true pleasure to look at.

Katariina Malmberg, Ines Vaittinen (ENoLL)

A Living Lab is an orchestrator of open
innovation processes focusing on co-creation of
innovations in real-world contexts by involving
multiple stakeholders with the objective to
generate sustainable value for all stakeholders
focusing in particular on the end-users.
Anna Sthlbrst, Botnia Living Lab

A Living Lab is a multi-stakeholder organization

set-up to carry out innovation projects that follow
the principles of open and user innovation and
focus on real-life experimentation.
Dimitri Schuurman, imec.livinglabs

A Living Lab Is a place where citizens, artists,

technologists, businesses and public sector
organisations can come together to co-create
ideas, tools and technologies that will address
local challenges. Its a place for innovation and
exploring new possibilities but where reflection
and evaluation are built into the working process
to make sure the Living Lab can be flexible and
responsive to the changing needs of stakeholders
and communities.
Penny Evans, Bristol Living Lab

For me a Living Lab is a creative space where

people from with many different interests
and backgrounds can collaborate in new and
imaginative ways. Living Labs are also creative
spaces for sharing technical skills and technical
spaces for sharing creative skills. This makes
Living Labs very special places.
Dave Carter, Manchester Urban Institute,
University of Manchester