Evaluation of eCat and Search Techniques

Please enter your participant code here:

Questionnaire purpose


This research project (Streamline) is examining how to improve on the processes of producing and searching for Learning Objects. The focus is on reducing the administrative overheads associated with both processes from the end users perspective. As part of this research we have been examining the use of eCat’s process for adding metadata to Learning Objects. Further details can be found on the Streamline blog.

Please could you complete this questionnaire as part of the eCat evaluation process and an examination of Learning Objects’s search techniques. This questionnaire is completely anonymous and all data gathered will be used for research purposes only. By completing and sending this questionnaire you are giving us your consent to use this data in this way. Thank you for your time and participation.


A. Demographic questions


Q1. Gender? Male Female


Q2. Age?



Q3. Country?


Q4. What is your highest attained education level?


Q5. To which faculty/department do you belong to?


Q6. What are your areas of expertise?


Q7. How long have you been teaching for?



B. Previous Experience

[TOP]


Q8. Have you created any learning objects prior to the eCat workshop?


Yes: go to Q9 No: go to Q12


Q9. Please list any other applications you have used to create Learning Objects?



Q10. Give a brief description of how you went about creating the Learning Objects



Q11. How would you rate the process of creating learning objects with these applications? (where 1=easy and 5=hard).



Easy


Average


Hard



1 2 3 4 5




Q12. Have you used any Learning Objects that you did not create yourself?


Yes: go to Q13 No: go to Q16


Q13. Where did you find the learning objects you used?



Q14. Give a brief description of how you went about finding the learning objects.



Q15. How would you rate the process of finding Learning Objetcs in these repositories? (where 1=easy and 5=hard).



Easy


Average


Hard



1 2 3 4 5



Q16. In your opinion how important are learning objects in developing eLearning? (where 1=low and 5=high).



Not Important


Average


Important



1 2 3 4 5



Q17. In your opinion how important is metadata within a leaning object? (where 1=low and 5=high).



Not Important


Average


Important



1 2 3 4 5



C. Evaluating eCat

[TOP]


Q18. How would you rate the eCat application in the following areas: (where 1=low and 5=high)
Poor Average Good
a. Finding and using help 1 2 3 4 5
b. Adding images and video 1 2 3 4 5
c. Adding metadata to media objects 1 2 3 4 5
d. Adding textual content 1 2 3 4 5
e. Adding metadata to the whole learning object 1 2 3 4 5
f. Completing the packaging process 1 2 3 4 5
g. Uploading the learning object to a repository 1 2 3 4 5


Q19. When entering metadata which did you prefer to use: the manual interface or the wizard


Q20. When using the manual metadata interface, (not the wizard) how easy or hard did you find it to use? (where 1=easy and 5=hard).



Easy


Average


Hard



1 2 3 4 5



Q21. Some of the metadata fields had default values already entered. Did you find this helpful?


Yes: go to Q22 No: go to Q23


Q22. Please give a brief description of why you did not find the default values very helpful.



Q23. Was there any particular aspect of the eCat process that you found very helpful?


Yes: go to Q24 No: go to Q25


Q24. Please give a brief description of those aspects of eCat that you found helpful.



Q25. If there is anything else you would like to comment on regarding the eCat application, please make your comments here:



D. Searching

[TOP]


Q26. Which of the following best describes how often you use search engines or facilities on websites?



 I am constantly looking for information in digital resources.



I look for information about once a day.



I look for information a couple of times a week.



I look for information a couple of times per month.



I don’t use digital resources.



Q27. Which of the following search methods do you prefer to use on individual web sites?



 Browse by categories or navigation links.



Simple keyword Search.



Advanced search within specific fields.



Q28. Which of the following best describes how often you use the advanced search features provide on search engines or web sites?



 I never use the advanced features.



I’ve never heard of advanced search features.



I only use them when standard search gets me no where.



I only use the advanced search features.



Q29. On average how many pages of search results do you view before you try another search or give up?



 I only look at the first few results.



I look at all the first page.



I will view up to 5 pages of results.



I will view up to 10 pages of results.



I will view all the pages regardless of how many there are.



Q30. Do you use Boolean operators or quotes in your search queries?


Yes: No: Don't Know:



Scenario: You are searching for a learning object to support students learning about Research Methods for their final year project/dissertation. The site you are using presents you with a single search box, (like Google).


Q31. What keywords would you enter into the search box? (Please type each word or phrase on a separate line)



Scenario continued: Still searching for Research Methods learning object, you notice that the site also has an advance search facility. This presents you with a series of boxes that search for the words you input within specific fields.


Q32. Rank the following list of search fields in order of which you would find them most useful in your search quest. The most useful in position 1 and the least useful in position 10.


a. Topic or Classification. 1 2 3 4 5 6 7 8 9 10
b. Keywords. 1 2 3 4 5 6 7 8 9 10
c. Date of learning object's publication. 1 2 3 4 5 6 7 8 9 10
d. Resource type (e.g. assessment, practical exercise). 1 2 3 4 5 6 7 8 9 10
e. Resource format (e.g. MP3, audio, text etc). 1 2 3 4 5 6 7 8 9 10
f. Difficulty level. 1 2 3 4 5 6 7 8 9 10
g. Author and/or contributors. 1 2 3 4 5 6 7 8 9 10
h. Associated institution. 1 2 3 4 5 6 7 8 9 10
i. Language. 1 2 3 4 5 6 7 8 9 10
j. Learning time. 1 2 3 4 5 6 7 8 9 10


Scenario continued: The search results give you a page of details about the learning objects that match your search query for a Research Methods learning object. This consists of a Title and an Abstract describing each learning object. What other information would you find useful in the search results?


Q33. Rank the following list of fields in order of which you would find most useful on a results page. The most useful in position 1 and the least useful in position 10.


a. Number of times downloaded. 1 2 3 4 5 6 7 8 9 10
b. Previous users, ratings. 1 2 3 4 5 6 7 8 9 10
c. Date of learning object's publication. 1 2 3 4 5 6 7 8 9 10
d. Resource type (e.g. assessment, practical exercise). 1 2 3 4 5 6 7 8 9 10
e. Resource format (e.g.. MP3, audio, text etc). 1 2 3 4 5 6 7 8 9 10
f. Difficulty level. 1 2 3 4 5 6 7 8 9 10
g. Author and/or contributors. 1 2 3 4 5 6 7 8 9 10
h. Associated institution. 1 2 3 4 5 6 7 8 9 10
i. Language. 1 2 3 4 5 6 7 8 9 10
j. Learning time. 1 2 3 4 5 6 7 8 9 10


Other Comments

[TOP]


Q34. If you have any other comments you would like to add about the process of searching for learning objects, please state them here:



Thank you for participating in this research.

The Streamline project is focused on producing a series of user centred tools to improve the process of developing, searching and storing learning objects. We are in the process of implementing a Learning object repository, using Moodle, as a test bed for these tools. If you would be willing to populate this facility with your learning objects, would like access to this resource or wish to participate in further evaluations, please enter your email address here.

E-mail address:

This information will only be used in the context of this research and will be kept independent of the questionnaire data and confidential at all times.

If you have any questions concerning this questionnaire or this research project please email d.a.wood@leedsmet.ac.uk


Please enter your participant code here: