The Software Sustainability Institute's Guidance for Running Online Training
This blog post was originally posted on the Software Sustainability Institute’s website based on lessons learned by SSI members.
Photo by Christopher Gower on Unsplash
Introduction
The following is guidance based on the Software Sustainability Institute staff and collaborators running online training events which have become more needed due to the lockdown caused by the COVID-19 pandemic. Steve Crouch has run afternoon training events online for the Scottish response to the Royal Society RAMP COVID initiative; Mario Antonioletti, Lucia Michielin, Aleksandra Nenadic and Giacomo Peru have been running Edinburgh Carpentries events online; Aleksandra Nenadic and Steve Crouch have run and facilitated workshops and instructor training events online. Rachael Ainsworth has excellent online event management experience especially during the recent delivery of Collaborations Workshop 2020 online. Shoaib Sufi has oversight of the Collaborations Workshop series of events and moved the selection of the Institute Fellows from a face to face meeting to a successful online event which has taken place for a number of years now.
Given these experiences, the Institute’s Community Lead, Shoaib Sufi helped document the conversation with the other authors around best practices for running online training events. Below are things to consider and actions to take before, during and after running an online training event.
Before the Event
Platforms, platforms, platforms
You may or may not have a choice about which online platforms you are able to use. Perhaps your institution uses and mandates Blackboard Collaborate, Microsoft Teams, or you have access to Zoom.
If you have a choice then ask others who have used the platform which they prefer. In any case familiarise yourself with the platform as part of the preparation for the workshop, as you (or a designated facilitator) will need to guide your delivery team and attendees around the system to use it effectively for online training.
As well as video conferencing you might consider the chat system you will use to keep everyone engaged. Some conferencing platforms have this built in (e.g. Zoom) but using standalone systems e.g. Slack might help attendees get help and have side conversations in a way that is less distracting and easier to catch up with than using integrated chat.
Collaborative note taking can also be useful - there are different platforms for this such as using Google Docs, Hackmd or etherpad.
The actual teaching material may also be hosted somewhere, e.g. for The Carpentries they have a lesson format which is web hosted.
Get the trainer/helper to learner ratio right
It is much harder to resolve technical issues online than at a face to face event, hence the need to balance the ratio of attendees/helpers differently. Be sure to size the event appropriately so learners get the attention they need.
- As a rule the ratio of one instructor/helper to four or five learners works for online.
- Higher levels of interactivity and assistance would require a ratio of 1:4 vs 1:5
- 15 learners might be trained well in an online setting requiring active participation from learners. Consider advertising for 18-20 places (as there are usually no-shows) and having two instructors and two to three helpers for a two day workshop.
- Seven learners might be trained well, if you advertise 10 places (again cater for no-shows), and you have two instructors/helpers.
- It was noted that some topics, e.g OpenRefine and SQL needed fewer helpers than Python or R, the latter two involved much more live coding.
- For training events not involving live coding, you may be able to train 20 participants well in an online setting - this is the usual upper limit when online instructor training workshops are organised and they typically require two instructors and no helpers.
Have an event facilitator
Online training needs extra facilitation; something which is not needed in a face to face event.The event facilitator will help make sure the virtual event is working well and people are aware of what is happening next and which resources to use. They can also help manage participants by muting those who aren’t speaking to minimise background noise.
It can be tiring and time consuming for the active instructor to monitor the chat, non-verbal feedback from participants and documents while they are presenting. An event facilitator can bring anything from the chat to the attention of trainers/helpers, freeing up the instructor to instruct and the helpers to focus on helping.
They can also explain logistical matters, e.g. explain the purpose and structure of using particular features of online platforms (e.g. breakout rooms).
Have learners at a similar skill level at the workshop
You can normally expect to get a variation in skill levels, but in online courses that variation is more apparent to the flow of the event. Variations in how quickly people get through exercises, material, or technical difficulties tend to be even larger.
If there is a mis-match in skill levels, e.g. you have an intermediate level cohort but happen to have someone attending who is at novice level then they can fill the schedule with many basic questions and issues that hold up the more advanced attendees.
Use a pre-workshop survey to gather learners’ backgrounds and gauge their current skill levels.
Plan your agenda for the online format
The agenda for an online event must have sufficient breaks for it to be effective and less tiring for those who are attending.
There are a number of model agendas one could use, for example for a two day or an afternoon workshop.
The two-day workshop:
A workshop that runs from 9am to 5pm for two days:
Day 1
- 09:00 to 09:30 - Welcome and any trouble shooting
- 09:30 to 10:30 - 60 minutes of teaching
- 10.30 to 11.00 - 30 minutes break
- 11.00 to 12.30 - 90 minutes of teaching
- 12.30 to 13.30 - 60 minutes lunch break
- 13.30 to 15.00 - 90 minutes of teaching
- 15.00 to 15.30 - 30 minutes break
- 15.30 to 17.00 - 90 minutes of teaching
- 17.00 - Close
Consider teaching for 60 mins in the last session to allow an earlier close at 16.30 and a less tiring day.
Day 2
- 09:00 to 09:15 - Welcome and any trouble shooting
- 09:15 to 10:30 - 75 minutes of teaching
- 10.30 to 11.00 - 30 minutes break
- 11.00 to 12.30 - 90 minutes of teaching
- 12.30 to 13.30 - 60 minutes lunch break
- 13.30 to 15.00 - 90 minutes of teaching
- 15.00 to 15.30 - 30 minutes break
- 15.30 to 16.45 - 75 minutes of teaching
- 16.45 to 17:00 - 15 minutes for any post workshop messages and close
Consider teaching for 45 mins in the last session to allow an earlier close at 16.30 and a less tiring day.
The half-day workshop
This could run in the morning (e.g. 9am to 1pm) or the afternoon (e.g. 1pm to 5pm).
Half day structure (blocks or 40 minutes teaching with 10 minute breaks and a longer break in the middle):
E.g. for an afternoon workshop:
- 13.00 to 13.20 - Welcome and any trouble shooting
- 13.20 to 14.00 - 40 minutes of teaching
- 14.00 to 14.10 - 10 minutes break
- 14.10 to 14.50 - 40 minutes of teaching
- 14.50 to 15.20 - 30 minutes break
- 15.20 to 16:00 - 40 minutes of teaching
- 16.00 to 16.10 - 10 minutes break
- 16.10 to 16.50 - 40 minutes of teaching
- 16.50 to 17.00 - 10 minutes for any post workshop messages and close
Things will take longer
Things are likely to take longer in a virtual setting than face to face:
- Factor in extra time for challenges
- E.g. a 10 minute challenge done face to face can take 30 minutes in an online setting.
- Factor in extra time to go through training material
- E.g. six hours of face to face material can take nine hours to get through online.
Build in extra buffer time for technical problems and people having not read or set up what you have told them to, since resolving such problems online can take much longer. Be prepared to troubleshoot during breaks or lunch time, in which case having longer breaks comes in handy and still leaves some time for an actual break.
Set break times and keep to them
Make the agenda with exact break times available as early as possible. This will allow people to plan their day and any caring responsibilities as much as possible. Stick strictly to the break times - make changes to your teaching plan if you have to on the day but not the breaks.
Make dataset availability easier
Encourage attendees to use local storage on their machines wherever possible to download datasets to and access them from. This will decrease any access problems and increase performance around any data access. For example, they should be discouraged to download any needed datasets to university file servers which they might have problems accessing due to network or availability reasons.
When giving instructions on where to download needed datasets from, consider using public repositories or cloud file systems for storing any needed datasets (e.g. Figshare, Zeonodo, Dropbox, Google Drive).
If there are any data privacy issues with the data you are sharing you will have to plan your workshop accordingly.
Have a pre-event rehearsal for the delivery team
Have a pre-event rehearsal for hosts, facilitators, instructors and helpers to help things run smoothly and on script on the day.
This is a good time to cover:
- The schedule for the delivery team vs the schedule for attendees on the day
- Make roles and responsibilities clear
- Confirm everyone’s technical setup is done
- Discuss tip and tricks so plans are not derailed on the day:
- How to turn notifications off and on for various applications
- These might be needed when not instructing but need to be turned off when they are instructing (e.g. if a private chat is being used between the delivery team to help the day run smoothly)
- How to avoid software updates to computers just before the event
- How to turn notifications off and on for various applications
Have a (mandatory) pre-event meeting for attendees
A mandatory pre-event meeting for attendees can clear up a wide set of problems which could derail your actual day so it’s well worth having (and mandating!). Even though the pre-event should be mandatory, a nice tone which conveys the purpose of the event should be used when communicating to attendees, e.g. “Come and learn about the platform and have your setup checked”.
Things which should be covered:
- Check their connection works, catch any audio / visual / bandwidth problems early.
- E.g. ask people to test their bandwidth (e.g. https:/ www.speedtest.net/ )
- If they have a slow connection you could advise them to turn their own video off on the day.
- E.g. ask people to test their bandwidth (e.g. https:/ www.speedtest.net/ )
- Learn about non-verbal feedback and any other features of the video conferencing tool you plan to use.
- Check that they have any pre-requisite software and dependencies installed for the session.
- Check that participants can share their screens or what to do if they have a multiple screen/device setup.
- E.g. pointing the camera of one device at the screen of another if need be for troubleshooting.
- Help troubleshoot any installation/setup problems before the training event.
- Run through some slides and your terminal windows with those attending to make sure they can see what you are doing.
- Some systems or setups for sharing might clip the bottom line/section of any screen sharing making it harder to follow along.
- Remind people about any technical steps they can take to make their experience smoother on the day:
- Cloning of repositories can take time, so do it before the day comes.
If it’s difficult for everyone to attend a pre-event meeting then alternatively drop in sessions could be scheduled to allow participants to get help before the event at a time that suits them.
Advice on physical setup
Giving advice on physical setup for learners can really help them get the most out of the experience.
In live coding workshops, following a screen share (with enough screen real estate to see what is happening), chat, reference material, shared documents and their own terminal window can be a real problem. There are setup tips which can help here:
- Suggest they have two monitors or two screens e.g. a monitor and an ipad or a laptop screen and a television as a second display (e.g. connected via a HDMI cable):
- Suggest a layout for easier working on the day, e.g.:
- Screen 1: Following the presenter, the chat, typing in code
- Screen 2: Following the notes, viewing/editing any shared documents
- Suggest a layout for easier working on the day, e.g.:
- Alternatively suggest one larger 27 inch + monitor with at least QHD resolution (2560 × 1440).
- Again, suggest a layout for easier working.
- Highlight the benefits and disadvantages of de-coupled screen setups
- It’s better to have two screen even if they are decoupled
- Connecting independently from two different devices means that some things become more difficult in terms of troubleshooting:
- Inability to copy-paste between e.g. an ipad and a laptop
- Screen sharing can become difficult - you might have to use the camera on one device to point at the screen of another to help with troubleshooting
- There are some ways to mitigate things
- Using the shared document to move text between the two systems
Manage breakout rooms or side meetings
Keeping attendees in the same online space or room for the entire event, especially if there is a fairly small number of them (e.g. 7-15 people) is preferred and from experience better for the flow and efficiency of the workshop.
For larger groups, platforms such as Zoom and Blackboard Collaborate allow breakout rooms, side virtual meeting spaces from the main meeting. If you are using this feature then there are some factors you need to take into consideration:
- Practise with the breakout rooms to see how they work and what features they have as they can be done in different ways, e.g. Blackboard Collaborate and Zoom both have the feature but it is not implemented in the same way.
- If you are using breakout rooms or side meetings to help people who are stuck, it can be difficult to catch up with the conversation in the main room.
- While in breakout rooms, people also lose access to the chat of the video conferencing platform and cannot see any slides being screen-shared in the main session.
- This is where using a Slack team for chat can be useful as it is independent of connectivity/drop-outs to any collaborative tool (e.g. Zoom).
- Be aware that it adds an extra application to be used and switched to and from by the participants; although its usefulness offsets this.
- This is where using a Slack team for chat can be useful as it is independent of connectivity/drop-outs to any collaborative tool (e.g. Zoom).
- Breakout rooms can cause loss of context so make it clear:
- What their purpose is if you are using them.
- If participants are placed in them for an activity, give them a set of instructions to follow and be able to refer to.
- Give them specific instructions on how they will be joined back to the main room.
- E.g. will this be automatic, at what time and is any action needed by them.
- How to keep up with the general workshop chatter (e.g. by using Slack).
- Setting and managing breakout rooms also adds extra time to the event (which will need to be factored into the agenda).
Share on a need to know basis
Share information about the meeting (e.g. connection details, shared documents) with those who are registered for the event. Do not publish the details for all to see and be clear to attendees about which information should not be shared (e.g. on social media). This is to protect your event from being disrupted by random individuals.
During the event
Use a videoconferencing code
A video conferencing code is a policy on how the meeting will use various features and modes of interaction in the online meeting in a consistent and accepted manner. Introduce and use a video conferencing code when running the meeting:
- Ask presenters to turn all notifications off on their machine - they may not be embarrassed by over sharing but these will be distracting to attendees.
- Ask people to mute if not speaking.
- Ask the event facilitator to manage participants and mute as needed (sometimes participants are unaware that they are unmuted and causing a distraction).
- Ask people to name themselves to use their full names.
- Helpers and instructors should add their role(s) after their names in brackets.
- e.g. Mario Antonioletti [instructor], Aleksandra Nenadic [helper]
- Introduce or recap how to use non-verbal feedback.
- Facilitators, instructors and helpers should be reminded who is monitoring non-verbal feedback and how questions and concerns will be routed from learners.
Use platform features to help coordination
Platform features can aid communication and coordination at event:
- Use a waiting room (e.g. in Zoom) so that hosts, facilitators, instructors and helpers are synchronised before letting learners into the virtual room.
- For example, in Zoom you can use non-verbal feedback via yes/no to mimic the green/red sticky notes used in the Carpentries to indicate how participants are doing and when they finish exercises.
Use collaborative documents to keep everyone synchronised
Use a collaborative document for the agenda, note taking and as another place to collect questions:
- There are a number of choices; a system that supports line numbering is essential to allow quick referencing for participants.
- Etherpad is an open source tool that allows line numbering which can be very useful (note these pads are public so best not to share any personal data via them, e.g. in the case their location is tweeted).
- etherpad.wikimedia.org for general use
- pad.carpentries.org/ for ‘The Carpentries’ community
- Google Docs are also very good.
- You will need to download an extension to allow line numbering
- Etherpad is an open source tool that allows line numbering which can be very useful (note these pads are public so best not to share any personal data via them, e.g. in the case their location is tweeted).
- Remind people that the chat (e.g. Zoom) can be ephemeral, or hard to contextualise after some time has passed (e.g. Slack, etc.) and to put things into the collaborative document if they think it’s useful for other attendees and if they want to preserve notes after the event.
Consider using Closed Captioning
Consider using closed captioning to help people review what’s been said and catch up if they have to step out with a helper or other reasons.
- Zoom supports closed captioning - a team member could be assigned to this but it would be an additional role.
- Zoom also supports auto closed captioning via third parties, but we have not tested these.
- Some suppliers are:
Get permission from attendees
If you are going to use or publish notes and resources created at the event then it’s important to let attendees know about this before you start.
Also if you are going to take any screenshots of the event or screen record the event it’s important to let attendees know and give them the option of turning their cameras off. It should also be made clear to attendees what these videos and photos are going to be used for.
Give permission to attendees
Learners need to know what type of learning culture you are creating, so it’s important to give permission for the behaviours you feel will help the effectiveness of the workshop. Interruptions in general should be accepted more than at face to face events because these may be the only way to gain the attention of the presenter to an urgent situation which they haven’t noticed.
So let learners know it’s ok and even expected to:
- Raise issues - so problems are addressed (e.g. in the chat or in the shared document).
- Directly interrupt - so problems do not linger.
- For example, if an instructor is using presenter mode (with their notes showing and slides minimised) for their presentation, it’s hard to see the information they need.
- For multi-day events, at the end of the day it can be useful to capture in the shared document what learners feel is working well and what needs improving. This could even be done per training session to allow adaptation by the delivery team.
Consider using a countdown timer
Instructors should consider using an on-screen countdown timer when sharing their screen for exercises with the learners. It can reduce the need for attendees to check how much time they have left, allow them to focus on working on the solution and let them know when the instructor will work through the answer.
After the event
Ask for feedback
It is best to ask participants to fill in a feedback form before they leave the event; otherwise the feedback rate drops significantly. It can be helpful to include questions regarding the online delivery of the event separately to the course content, so you can judge this aspect on its own without having the two aspects conflated in the feedback.
Further useful resources:
From The Carpentries initiative
- A brief blog on tips for teaching online
- The Carpentries recommendations for teaching workshops online
- Easing installation requirements for online workshops
- A discussion forum tagged for online training, run by The Carpentries
- An example of an online Data Carpentry event series run at the University of Edinburgh as part of the Data Skills Workforce Development funded by the Scottish Funding Council (SFC)