5.1. Money is Always Important
In both workshops, the issues related to Financing were given a high level of importance. The level of importance may also help to explain why at both workshops someone chose to look at solutions for issues related to financing, even though many were thought to be outside their control. Examining the issues more closely, participants are not just placing the blame on funding agencies—Some issues also show reflection, such as the need to make smaller steps or “Unrealistic project expectations/goals” by project partners.
Particularly the issues in the category
Marketing may point to unrealistic expectations on the part of project teams with regard to commercialization—The funding agencies aim only to “foster the emergence” also through “localization” [
31]. Issues from the poster indicate that in practice, going to market is in some cases an add-on and not considered from the start. For example, some items from the interactive poster: “Taking into account commercial aspects from the very first moment of concept” or “Product definition driven by research topics and tech push (should be user demand driven).” Both of these were rated in the control of projects. It is interesting to note, that the person who added the issue about commercial aspects was from a funding agency, so developers may not be sufficiently aware of this issue.
If we want to get AAL systems to the market, funding issues are of primary concern. In workshop 1, all issues in the
Financing category were rated of highest importance and most issues given highest importance (8 of 13) were in this category (with 3 more about
Marketing). In workshop 2, more than half of the issues related to
Financing were given the highest rating (7 of 12) and all issues given the highest rating were from this category. Funding problems can result in entire projects being stopped, which was demonstrated by the case studies done previously [
20,
21]. But they can also lead to people changing the focus. One of the case studies that led to this study being done found that due to costs associated with installation and maintenance a company switched from focusing on people in their own homes to those in care homes; this in turn affected the needs to be addressed in subtle ways and ultimately may have affected the success [
21]. Thus, funding agencies may want to reconsider funding guidelines to better support this phase, for example, by providing more flexibility, as suggested by the participants of workshop 1 (see
Figure 7), even if this just means delaying the time at which results are submitted with no additional funding, or providing new funding schemes aimed at this.
Once systems are on the market, to be successful they obviously also need to be purchased. The issue “Benefits unproven” indicates there is currently little impetus to invest in these systems. Also other recent research supports the need for more evidence about their effectiveness [
32]. Thus, more research is needed with regard to this, however, this is difficult as “Benefits [are] hard to prove.” Another complication is that the aspects that need to be investigated or proven vary depending on who will pay for these systems, whether it is individuals, care agencies or social systems. This determines what is deemed as more important: an extension the time people can live independently, a reduction in the number of visits from professional carers or a reduction in the number of accidents requiring emergency care, just to name a few possibilities. Also others report that there is no agreement about whether savings or effectiveness are most important [
33] (p. 73). While the case studies conducted and some of the systems described by workshop participants were aimed at private customers, i.e., older people and their families, others have indicated that these systems could amortize for social insurances in just a few months [
34].
Participants in one workshop felt that the lack in value related to the fact that the technology emerged from the funding, e.g., aimed at keeping people in their homes longer [
31], rather than emerging from the needs of older people. This is an important issue. If these systems are primarily targeted at saving money for social insurances and features for older people are only added to induce people to agree to having them installed, perhaps we should not be surprised that they are not popular with older people. Older people not only are the ones who are faced with these systems on a day-to-day basis, they are concerned about other aspects, including how useful the systems are for themselves and also whether they will lose existing care structures [
23]. As identified by the participants in these studies, from the point of view of older people small aspects about the product, even just an LED that is too obvious, can be problematic, as the appearance is an important factor, even if the primary goals of the system are related to health [
35]. If financing programs are framed to save money on care, an LED is a good feature, as it gives feedback about whether the system is still working and so developers may not give sufficient consideration to the fact that it may make the home of the person feel more like a hospital and so affect acceptance. Thus, user needs and preferences must be given more weight—Something that may be impeded by current grant structures that require project goals be set before a user needs analysis has been done. Still there is hope for future projects—Participants believe there are things that can be done both by funding agencies (workshop 1) and companies (workshop 2).
5.2. Users and User-Centered Design
Since the authors come from the area of Human-Computer Interaction (HCI), they expected to see issues related to user-centered design (UCD) and participation. In fact, several issues relating the
Users were confirmed, though only one was rated most important. The relatively low importance given to these issues may be due to the fact that older people were not consulted. It may also be due to the fact that many of these issues were considered to be in the control of teams. Furthermore, it has been found that most AAL-JP projects do include users in some form [
3]. Since people are including users, they may think there are fewer problems than there actually are.
In contrast to the perceived importance given to these issues, two of the five “solutions” specifically mention that teams need to be more user-centered. This fits with the findings of the evaluation of the EU-wide funding program AAL-JP: “Although users are well represented within projects, they are not sufficiently integrated and overall effective user involvement is still sub-critical” [
3] (p. iv). The solution developed in workshop 2 for the issue “Developers are young → older people stereotyped” indicates that participants felt software engineering education has failed to make developers really look at the needs of the users, further evidenced by the issues “Don’t really understand needs of the user group” and “Still little inclusion of users beyond testing.” Furthermore, the solution developed in workshop 1 (
Figure 8) demonstrates the large number of stakeholders involved, many of who have different goals. If users are included only for testing, the design and features may be based on the perceived needs described by others.
Given the problems understanding the needs of older people, it is perhaps surprising that the diversity of older people mentioned elsewhere [
19,
22] was given only medium importance by participants. Although this issue was confirmed by multiple sources, it was not one of the most important issues, perhaps because it is not something people really reflect on, because it is widely known and something that just has to be dealt with. However, one of the solutions developed provides a potential way of dealing with this diversity: developing smaller add-ons rather than large comprehensive products. This could support matching the more specific needs of people by tailoring the features incorporated, i.e., only those desired and needed. This would also address the barrier identified elsewhere, that people are concerned about the suitability of these systems for everyday use [
13]. Allowing flexibility with regard to the functionality may, however, create new challenges, as it can compromise reliability and requires all possible configurations be tested if the systems relate to security [
36]. Moreover, some sort of support or advising may be needed to support people selecting appropriate features.
Overall, the issues in the category
Users indicate available resources and methods are not being tapped by developers in order to understand older users and their diversity. Many developers have access to people with more experience with older users and family members who are older. Furthermore, funding programs have financed the development resources specifically for AAL. This includes a set of personas developed to help raise understanding of and empathy with older people [
37] and a toolbox of UCD methods appropriate to AAL projects [
38]. Despite their experience in AAL, even the participants of workshop 2 did not seem to be aware of these resources. In fact, a study conducted in Austria found projects developing AAL technology tend to use more general methods like interviews [
39], rather than methods specific to user-centered design, such as shadowing, that are recommended by this toolbox of methods for AAL [
38]. This has been found also by other studies on software development done elsewhere in Europe [
40]. When selecting UCD methods the features of a project need to be considered, e.g., the amount of time pressure, the skills of the people and also the amount of access to users available [
41]. Furthermore, special methods may be required for working with older people [
42,
43].
Even if developers do know and apply appropriate methods to gain understanding about the needs of users, there may be difficulties applying these methods. A recent survey of projects developing these types of technologies found that at least in Austria older users are often not included in the early phases of AAL projects and only included in the final evaluation [
39]. This is especially a problem since one issue identified was “Finding people for long-term tests,” something that has been reported in a publication about projects including partners from different European countries [
44]. Instead participants reported that other stakeholders are included in the development instead and that “Needs of carers take precedence”. This is critical, as others have found the priorities of older people may differ substantially to those of carers, for example, by putting more importance on privacy and aesthetics than on safety [
22,
36,
45]—Also the participants reported that privacy is an issue when developing AAL technologies. Not giving older people sufficient voice in products and using them only to check if things work as expected, may impact the success of the systems.
The problems may also arise later during the development, after needs analysis. This same study about projects in Austria indicates there are problems communicating with the developers [
39], meaning that the user researchers and even designers may know the needs of the users but the information does not reach the developers, who may need to make design decisions during the project. Instead, it may just be that teams are distracted from user issues as technical issues take over, as has been found in one of the case studies leading up to the workshops [
20]. The issues from the workshops, e.g., “High reliability needed in diverse situations,” that “Reliability takes over—But usability important to acceptance,” support this may indeed be happening. Reliability is certainly an important aspect for these types of systems [
13,
46]. To help ensure that the users and their needs do not get forgotten, techniques like those used to ensure personas are kept alive may help, for example, using the user stories to evaluate functions considered, or regular e-mails with new facts relating to the users [
47]. Good project management and frequent team meetings could also help remedy these problems in the development by reminding the team of the project goals. At the same time, these measures could address the communication issues that were identified in the category
Project, which were thought to be important to success.
Finally, there are challenges related to the evaluation of the systems with users. On one hand, it is advantageous if evaluations are done early in the development while changes are still possible, however, in practice “Technical prototypes (are) too complex for field tests” making it difficult to get feedback. As documented elsewhere it may also be difficult to explain intangible aspects, such as those related to sensors and activity monitoring, to older people [
19]. As reported by the participants, people involved in long-term tests may need a lot of support initially. This should not deter from doing evaluations with older people as they are particularly important with AAL systems, also because participants reported “Changing needs of older people,” which could mean that the needs of the people involved may change even in the course of a single project. Having longer periods of test may help uncover a realistic range of changes people experience and give a more accurate view of systems to ensure the resulting systems meet the real needs of older people longer term. Also others have found that the needs of older people change with regard to these types of systems even after six months, as people may experience physiological changes and also gain confidence with the technology [
48].
The issues relating to the users indicate the need on the one hand to understand more precisely what is happening in projects, for example through further case studies and on the other to spread information about UCD methods in the AAL community but also to gather more evidence regarding their suitability.
5.3. Methodological Aspects
The methods aimed to explore issues projects have faced. It included only a limited number of projects and participants. As such, the results do not include all issues experienced by projects, or even necessarily the most important ones. Furthermore, although the issues presented here were reported by multiple people involved in projects, there are other projects that have not experienced some or even all of the issues identified.
Input was gained from at least 30 participants with experience in the development of AAL systems. Although this is not a large number in comparison to more general studies investigating risks in projects [
16], we are dealing with only a small field. The specific invitations to workshop 1 ensured that a wide variety of participants and people with a lot of experience were included.
The participants may not be representative of the community as a whole. Thus, the results only say whether another participant had the problem, not that no one else did or how frequent it is. The focus was placed on the European context. However, since most participants in the workshops were from Austria, it may mean that some of the issues from the workshops and solutions are specific to this context. But this may have less importance than it first seems, certainly with respect to the AAL-JP, Austria has been one of the countries involved in the largest number of projects [
17]. Furthermore, over half (at least 8 of 15) of those workshop participants from Austria had been involved in projects that included partners from several European countries (e.g., those funded by the EU funding programs AAL-JP or Interreg). Conducting the interactive poster at a European conference in another country ensured views also from researchers based in other European countries were included, so that in all approximately half of those participants contributing to the issues were based in other countries.
In workshop 1, all issues were assigned either the highest or medium level of importance—None were included in the lowest “band.” In the second workshop, both groups had issues in all bands of importance. Thus, the participants in the workshop 1 may have simply given things a higher priority but they may also have focused on those things that were most important, because they were invited specifically to share their knowledge.
From the text on a sticky note alone, it can be hard to know what participants meant with some issues, particularly with the interactive poster, where no categories were available to help interpret. If an author was nearby, in some cases additional information could be gained, which was also useful when for checking for duplicates. For the workshops, the audio recordings and categories supported understanding what was meant.