8 min read

Cities around the world are increasingly using Civic Tech platforms (‘e-Participation’) to actively engage citizens in plan-making and development projects. Yet what is the real influence of e-Participation in both planning decisions and processes?

This is the one-million dollar question that is valid for all currencies and stakeholders – whether you are a professional, tech vendor, researcher, consultant, community leader or an aficionado who is simply interested in urban affairs. Investing in effective public participation promises to improve decisions and deliver greater all-round community value. Can e-Participation live up to these promises?

Just as e-Participation is no a silver bullet to designing and managing sustainable places, the evaluation of e-Participation remains fraught with challenges.

This blog post is a companion post to the ‘Objectives and design of e-Participation in urban planning‘. The findings I share here are based on the same data, namely: interviews and survey responses from 80+ planning professionals about 60+ projects, concerning 25 e-Participation platforms. I also interviewed representatives from 13 Civic Tech start-ups.

The post begins with a presentation of some general challenges in assessing the effectiveness / impact of e-Participation.

I then share the findings from my own research

General challenges for assessing the impact of Civic Tech

Assessing impact measurement in the general field of Civic Tech is a perennial challenge. Matt Stempeck and Micah L. Siffry from Civic Hall, US, presented the following 10 main hurdles at TICTeC 2018:

1. Everyone uses different metrics to assess participation on Civic Tech platforms – different local authorities, consultants, tech vendors, researchers all evaluate participation differently
2. The sharing of impact assessment is irregular
3. Most projects (still) don’t reach most people – this can be due to digital divides, a democratic deficit, a participatory deficit, etc.
4. Different constituencies want different metrics – e.g. API development vs real impacts in society
5. Assessment of Civic Tech outcomes is seldom linked with financial investment – which begs unanswered questions about ROI, SROI, and so on.
6. No evaluation relative to the wider/macro environment is usually made
7. Quantitative metrics can miss the point (i.e the objective and purpose) of online participation altogether
8. Qualitative cases and insight are often too biased
9. Causality is hard to prove concerning social environments, if not impossible
10. “Our externalities may eat us all” – do Civic Tech projects actually serve civic purposes and build real communities? Or the careers and interests of those leading the projects?

Critically, organisations may lack the resources to assess e-Participation. Furthermore, the ten challenges listed above open up questions more than they answer them, which is perhaps the very nature of public participation in conversations that matter.

Although the above concern a wide range of Civic Tech that do not necessarily relate to spatial planning, they do provide an engaging prelude to the findings from my own research.

Influence on planning decisions and processes – the view from planning professionals

The findings about e-Participation / digital participatory platforms (‘DPPs’) are based on interview and survey responses from 80+ planning professionals across Europe, North America and Australia.

The findings are self-explanatory. I briefly discuss the main points.

Influence on planning decisions

First and foremost, it is difficult, if not impossible, to isolate the influence of e-Participation / DPP platforms. As mentioned in this companion post, the platforms are often used with other tools. Their use is also inseparable from the wider context, and they may constitute only source of evidence to be considered in planning processes and decisions. Particularly, because planning typically takes place on such a long timescale and needs to consider and balance multiple sources of evidence, it is difficult to establish a linear influence between a specific e-Participation project and planning outcomes.

Their perceived influence is linked to their perceived value. This depends on how deeply / actively citizens can participate in the planning process – whether they are actively involved and invited to collaborate or co-deliver solutions, or whether their only asked to provide feedback. Value is therefore linked to the quality of participation – are the comments and other input on the platform of high quality and relevant for the planning context? Also, how representative is the input on the platform, and how much demographic and other data is available to understand which groups are being engaged?

While basic metrics about participation are routinely available (e.g. number of participants, number of comments/ideas submitted, common website analytics), formal evaluations of e-Participation are typically qualitative. By and large, there do not seem to be established procedures for in-depth assessment of individual tools for public participation, or resources allocated for that specific end.

Finally, objectives and expected levels influence are interlinked and open to interpretation – what might count as ’empowerment’ for one respondent might be viewed as involvement or collaboration by someone else. Perceived levels of influence are therefore contextual rather than absolute.

Influence on workflows and processes

These consist of ‘key adoption factors’ and ‘key factors for effective use and integration’.

Regarding the adoption of e-Participation, the cost of a DPP (whether proprietary or configured from Open Source code), the range of platform functionalities and features, and the intra-organisational capacity to use them in planning are essential. Likewise, the willingness to procure and use a platform often stems from a political ask and requires a general working culture that is open to what citizen participation entails, namely: to consider the views of residents and their capacity to influence the decisions that will influence them. Lack of support for e-Participation could also be investigated by virtue of organisations not adopting them in the first place, which would deserve a specific research project in itself.

A related post about procurement and lifecycle considerations is also worth reading (it’s an 8 minute read).

Regarding the effective use and integration of e-Participation, the findings mainly revolve around available resources (in-house skills and staff availabilty, guidance about e-Participation and public participation more generally, and room for collaboration with colleagues at other departments within the organisation). These relate to the capacity to manage citizen expectations effectively: launching a platform will create expectations of different kinds, which can be difficult to predict or even plan, particularly for first-time clients of DPP platforms.

Finally, the most effective use of DPP platforms takes place through continuous learning by doing and experimentation over the course of multiple consultations. This can also involve close relationships with software providers which may have strong participatory democratic and planning ethos, and go the extra mile in supporting clients achieve their objectives with a long-term perspective on influence and community impact.

Professional networks and communities of practice can provide peer-learning and good practice insight from their own experience, which can help both late bloomers and pioneer adopters of DPP platforms. Among other such professional networks, can cite the MetaDecidim community of Decidim software users, as well as networks of participatory budgeting professionals, such as the Participatory Budgeting Project in the USA, or the French networks of participatory budgeting professionals (coming together slowly) and community engagement professionals. Various non-profit and public organsiations are also available to support local councils and disseminate good practice among their members, such as FutureGov in the UK, or the Swedish Association of Local Authorities and Regions (SALAR – SKL). Through its TICTeC events and ‘Show and Tell’ sessions, the Civic Tech software and research-based consultancy mySociety also provides opportunities for UK-based and international actors in the field.

The view from software providers / Civic Tech start-ups

Thanks to their cumulative experience of dozens of projects (if not hundreds and thousands), the responses from 13 Civic Tech start-ups nicely augment the insight from planning professionals presented above.

In essence, it is up to clients to deploy the DPPs and integrate the input into planning decisions and processes in the way the see most fit. Effective marketing and promotion of both the planning and related consultation projects are essential. Influence denotes both processes and outcomes. Engaging people in the conversations that matter in planning is a long-term endeavour. Planning outcomes might come years after e-Participation has been conducted.

While DPPs reduce the net cost of engagement per participants compared to in-person methods, client organisations should allocate sufficient resources (staff hours, skills, budgets), which requires political support and a supportive organisational working culture.

Perennial institutional hurdles mainly concern ‘engagement divides‘ – the fact that residents often are not interested in planning or do not know how to participate, or are skeptical that they can actually influence decisions. This insight is also echoed in the Engaging for the Future report by the start-up Commonplace (January 2021). Additionally, digital divides here denotes the fact that people cannot access, or have no interest in using, digital devices for civic purposes. An additional unresolved challenge on a ‘glocal’ scale (both local and global) is wavering public trust in local government. Trust is not project-based and can never be taken for granted, it builds over time and can be nurtured through such powerful tools as e-Participation platforms, among other tools and methods. Toward this end, providing high quality information is key. People might end up participating after seeing a local authority consistently tries to engage residents in a clear and compelling way.

Last but not least, the open activism and advocacy ethos of many Civic Tech start-ups is instrumental to fostering planning environments and practices that help to raise resident’s participation in the decisions that influence them. This activism is often coded in the very design and features of the platforms, and sometimes also at the heart of creating and facilitating professional communities of practice that disseminate good practice and enable peer-learning.

Key implications

This post closes by considering three main implications from the finding in this post and in the companion post about the objectives and design of e-Participation.

All the investigated factors (organisational, social and technical) are interlocking and highly contextual. Therefore, what works in one project or for organisation might not work elsewhere. Furthermore, it means one needs to tackle all issues to achieve effective e-Participation. Because outcomes

This requires ample learning by doing, including gaining permission to make mistakes from elected officials and the public alike. The same applies to new Civic Tech start-ups on the block, which is why many start out by providing free services or discounted fees to their niche market. Starting small will also mean one can learn to fail at a low cost to build capacity and confidence. One can then gain further experience and trust to steadily grow and better evaluate impact over time.

Only collective knowledge and wisdom can transform information and data into a high quality evidence base and better decisions. Workflow automations, integration and reliable metrics are key for leveraging and measuring the impact of e-Participation in planning. However, the human heart and mind trump information and data overload, not to mention that the quality of datasets is often irregular. The quality of citizen input might also be unusable if the quality of information about planning is unclear, overwhelming or inaccessible.

These implications illustrate the recursive quality of progressive planning and community engagement practice. Experimentation and reflection in action are the stuff of innovation. Effective e-Participation requires ecosystems of tools empowered by collective learning, civic and spatial literacy, and capacity for transformation. At its core, e-Participation performs best if augmented by in-person, ‘boots-on-the-ground’ engagement methods, many of which are highly innovative and can reach out to those who are unable to take part in the conversations otherwise matter to them. Planning for a better future demands no less than rich ecologies of participation, and the collective resources to bring them to life.