The coronavirus pandemic moved the bulk of youth work activities online slightly over a month ago. Managers, decision-makers and funders soon wanted to know how this change affected the effectiveness, quality and impact of youth work. Do we meet as many young people online as we do face-to-face? How good is the quality of online activities? What results can we achieve through online activities?
It is important to compile statistics on activities to make them transparent and enable their assessment and further development. Statistics primarily help us answer the question about the benefits of activities. Therefore, it is essential to consider what you compile statistics on. Which indicators depict our success or the achievement of our goals? What kind of information can we use as the basis for developing our activities?
At the beginning of the state of emergency, our purpose was to move as much of our youth work as possible online. In this case, information about the number of young people reached through online services was an adequate indicator. As such, indicators also steer and encourage youth work to get active and take action in the best case. For example, by measuring the number of social media updates, we can encourage the work community to more regularly produce more content. Now that online youth work activities have been running for a while, statistics help us make a deeper assessment of the activities and analyse, for example, the quality of interaction.
We should keep in mind that being active online is not our ultimate goal, so there is no point in measuring online activity excessively in the long run. What is of greater importance is determining what youth-work goals we want to reach online – and how we can measure our achievement of these goals.
Based on what statistics, can we tell whether our activities:
- meet the goals set for us?
- have reached our target group?
- are of high quality?
- answer the wishes of young people?
- answer the needs of young people?
Since online activities’ goals differ depending on the organisation and form of work, there is no single, recommended method for compiling statistics on online youth work. Like other youth work, the methods and needs of statistical compilation depend on the organisations’ goals, target group, methods and services. In terms of the objectives and methods, youth work carried out on Discord and youth work on Instagram may differ as much from one another as youth work carried out in the street and youth work carried out in youth centres.
Online services provide tools for automatic statistics compilation
It is relatively easy to collect information about the number of visitors, views and reads, the time spent with a service, likes and other reactions directly from the online service. The nature of the service determines what we can collect statistics on and what solutions we can use in the compilation. For example, you can install a StatBot on your own Discord server to enable the collection of data. Take time to examine what statistics are available in your online services and look at them regularly.
Instagram, for example, provides a lot of information about visitors, especially if your organisation has a corporate account. You should at least follow your Instagram visitors’ age group to determine whether you are reaching the right target group. Visitor data also reveals whether you reach the right regional target group. You could also have a look at what social media channels show regarding the spread of posts. For example, has someone found your post to be so useful that they have stored it on Instagram? How many users have shared your Facebook post, and what do YouTube views consist of?
However, do not fall in love with statistical tools. Much of the statistics provided by the services are fun information, but they do not directly indicate whether young people have been really reached and what the quality of online youth work has been like, not to mention what kind of an impact the activities have had. The statistical tools provided by social media services have been created with communication, not youth work, in mind. If you only rely on statistics provided by social media, you will not be measuring how you achieve the core objectives of your own activities.
An excessive focus on social media statistics may lead to your activities being steered by such statistics instead of youth-work indicators. However, it is not always easy, nor relevant, to differentiate between communication and youth work in the online environment. We must make deliberate decisions regarding the statistics we follow and how we interpret them.
The importance of specification and guidelines
It is important to ensure that all employees understand the significance of statistics and their compilation principles. This calls for joint discussions and the preparation of guidelines to define, for example, how online encounters are defined. Is a like given to an Instagram post or the number of views of a YouTube video enough? Does the same definition apply to all online environments used for youth work?
How about the young people who are quiet participants in online discussions, those who “walk through the doors”? How do we treat them? After all, it isn’t easy to show that you are actively listening to someone online. The number of reads and views provide some information, and a young person who has viewed an online video or read the comments posted by other young people may have learned a lot to support their growth – the same way that in face-to-face situations, the quiet ones can participate by listening. However, we cannot take it for granted that everyone who has seen an update has acquainted themselves with the content in detail.
Another thing to keep in mind is that if for statistical purposes, online encounters are equalled with face-to-face encounters at youth centres or other similar places, some of the nuances typical of online interaction will not come out. Since strict definitions are not well suited to online services, it would be a good idea to discuss the definitions in the work community. The definitions must also be modifiable to adapt to the introduction of new operating environments.
What other ways are there to collect data?
When you have enough experience with online activities, you should begin to compile information about the quality of activities. The statistics provided by online services are not adequate for assessing all the aspects necessary to develop activities. Of course, the popularity of the service, as shown by the number of views or another metric, indicates its quality. Still, to carry out a more detailed analysis of quality, you need feedback from both young people and youth workers, peer assessments, self-assessment or an outside evaluation. Some of the easiest ways to collect feedback from young people include using a feedback field in Insta Story or creating a simple feedback survey that young people can complete on Discord or another similar channel at the end of the evening.
To develop the quality of youth work, many municipalities have long been using a youth work auditing and self-assessment model, including a special set of assessment criteria for online youth work. The assessment and development of online activities can also be carried out jointly with the target group and interest groups with various interactive methods.
When we talk about positive change achieved through goal-oriented activities, we talk about impact. It is challenging to measure impact because to prove that change has taken place, we usually need time, long-term monitoring and attention to many different factors. To monitor impact, we most likely need to combine many different sources of information and data collection methods. However, it is not impossible to measure the impact of online activities. Especially when activities are running at full speed, it is worth considering how to measure and make visible their impact in both the short- and long-term. The “Hyvän mitta” website, focused on the measurement of impact, offers a wide range of support material for discussions about impact.
Will artificial intelligence interpret statistics in the future?
When a great deal of data on the use of online services begins to accumulate from various sources, analysing it may seem an onerous task. Human reasoning is not always capable of distinguishing all the information important for the development of services. The more statistics we compile, the more frequently we should update our services accordingly because data grows sour if it is left unused. In other words, decisions based on statistics should be automated before long. This is where prescriptive analytics and artificial intelligence enter the picture.
In the future, various solutions utilising artificial intelligence will most certainly be used in youth work as part of services and as information collection tools in service development. It takes time and money to develop solutions based on artificial intelligence to meet youth work needs. Therefore, solutions should not be developed to meet a single organisation’s needs but rather more widely on a regional and national basis and across sectors. Cooperation is also needed because the data-based development needs in youth work services may be of the kind that no organisation can solve independently.
If artificial intelligence and the opportunities it offers for youth work aroused your interest, have a look at the related video (Finnish only) produced by Verke or learn more about the world of artificial intelligence in Verke’s latest publication focusing on aspects that youth workers need to know (Mitä nuorisotyön tulisi tietää?, Finnish only).
