Reflection & Evaluation

A reflection on the design process and a model for future evaluation

Reflection

Overall the process of proposing a design solution to the identified learning space problem was challenging and rewarding. However, there were sever key constraints which limited the success of the process. These centre on late to the lack of team during the ideate phase, an inability to fully consult with relevant stakeholders, and restrictions on my ability to complete the process.

The Design Thinking Process positively challenged me to avoid focussing on the ‘single, best solution’ and rather to generate a wide variety of innovative options to be further developed through testing and feedback (Stanford Design School, 2010). Unfortunately, just as the user needs analysis undertaken earlier had limitations -- access to key stakeholders and the time and resources to fully engage with a wide variety of users – the ideation phase of this design process was less than ideal. The requirement to ‘(h)arness the collective perspectives and strengths of your team’ assumes that one has a team to call on (Stanford Design School, 2010). As a ‘team’ of one I lacked the diversity of experience which allow for truly innovative solutions to be proposed. Although I could not access the ‘synergy of the group’, I attempted to mitigate this with informal discussions with personal and professional contacts, including our school’s Information Technology support person, my subject area Head of Department and several colleagues. My partner was also very willing to bounce ideas, although I suspect he will welcome the forthcoming change in dinner table conversation! Furthermore, I welcomed the directive to ‘embrace misunderstanding’ and surround myself with ‘inspiring related materials’ by consciously cultivating a positive and open frame of mind and reading widely on website and LMS design that were admittedly beyond both my likely budget and existing skill set.

The focus on inclusive design meant meaningful consultation with the affected groups would be essential, which was outside the scope of the current project. There are many groups and individuals whose input would have been invaluable, both to better understand user needs and as part of the ideation team. These include:

Cultural community liaisons (Vietnamese, African, Pacifica, Indigenous
Indigenous community engagement coordinator
Head of Special Education
Subject area coordinators
Heads of Department
EALD coordinator
Student Council

In addition, I found it particularly frustrating to not have the opportunity to prototype, test, and improve the proposed design. I further maintain some reservations about the practicality of aspects of the process within the scope of the Teacher Librarian role. Time and budget constraints, the lack of a dedicated team, and the pressure that consultation would place on stakeholders in terms of their limited time and resources.

Despite these limitations and frustrations, applying the Design Thinking process and engaging with theories and concepts related to design best practice in my context was ultimately enlightening. I have a broad grasp on best-practice learning space design theories and concepts and the process of Design Thinking. Importantly, I am confident that I could effectively advocate for change in the library space by offering robust design solutions.

References

Finley, T., & Wiggs, B. (2016). Rethinking classroom design: create student-centered learning spaces for 6-12th graders. Rowman & Littlefield.

Gibbons, S. (2016, July 31). Design Thinking 101. Nielsen Norman Group. https://www.nngroup.com/articles/design-thinking/#:~:text=The%20design%2Dthinking%20framework%20follows,say%2C%20think%2C%20and%20feel.

Stanford Design School. (2010). An introduction to design thinking—Process guide. In Hasso Plattner Institute of Design at Stanford University. Retrieved from https://dschool-old.stanford.edu/sandbox/groups/designresources/wiki/36873/attachments/74b3d/ModeGuideBOOTCAMP2010L.pdf.

The following is a proposal for an evaluation plan that demonstrates how the impact of the learning space will be assessed. The space was intended to encourage the independent development of digital literacy skills by offering a relevant, engaging and accessible LMS and reimagining the library as a learning commons. The CIPP (Context, Input, Process, and Product) model of evaluation could offer a useful framework for ongoing improvement of the design and provides a broad enough scope to be applicable to a digital space (Tan, Lee & Hall, 2010). The focus on clearly contextualising the scope of the evaluation would be especially important as the stated goals of the space are arguably quite subjective. A combination of both quantitative and qualitative data should therefore be collected in a range of formats to attempt to identify both actual usage, and user and stakeholder perceptions of the space. Furthermore, the evaluation must remain open to both unanticipated opportunities and unexpected measures of success (Johnson & Lomas, 2005).

Quantitative data

The LMS has powerful reporting and analytics tools built into the cataloguing tool. Data captured will show both staff and student use. This will allow easy analysis of page visits, unique user visits, and resource usage. These are highly customisable and automatically presented in a number of visual formats. This will allow actual usage to be effectively evaluated. The quantitative data could also be used to identify a core group of frequent users and non-users to target specific and response evaluations to these groups.

Qualitative data

Qualitative data will be necessary to add richness and nuance to the evaluation. The school population frequently use online surveys to collect relevant data, so surveys which includes a variety of scaled and short response questions to identify perceptions of the learning space and suggestions for improvement would likely be well received. Padlet may also be an appropriate forum for ‘comment wall’ style feedback and offer a different format for responses. Qualitative data will need to be collected from groups beyond the student users to ensure the full scope of the learning space is evaluated. The perceptions of the leadership team are important for ensuring ongoing support of innovation. Teacher perceptions and usage of the subject-specific resources must be measured to encourage the use of the space within classrooms and ensure that the space is facilitating curriculum delivery. The cultural liaisons and other representatives listed above will also offer unique perspectives into the uptake and experience of different groups. With all of these groups, informal discussions and Professional Learning Teams would create a valuable source of data for evaluation.

Accessibility evaluation

Regular evaluation of the accessibility of the site will also need to be undertaken to ensure the needs of new and existing users are actually being met. Furthermore, emerging technologies will inevitable alter the framework and content of the space, requiring ongoing evaluation of the structure of the space to ensure the interface continues to meet the changing needs of users (Manzoor & Hussain, 2012).

Evaluation

References

Johnson, C. & Lomas C. (2005). Collecting Data https://er.educause.edu/articles/2005/1/design-of-the-learning-space-learning-and-design-principles

Manzoor, M. & Hussain W. (2012). A web usability evaluation model for higher education providing Universities of Asia. Science Technology Development, 31 (2), 183-192

Tan, S., Lee, N., & Hall, D. (2010). CIPP as a model for evaluating learning spaces. https://researchbank.swinburne.edu.au/file/b9de5b45-1a28-4c23-ae20-4916498741b8/1/PDF%20(Published%20version).pdf