TopicsTeacher Interview

Practice of Basic Informatics: Providing Accurate Feedbacks to
a Large Number of Students

Associate Professor Hiroyuki Sakai and Program-Specific Junior Associate Professor Masako Okamoto, Center for the Promotion of Excellence in Higher Education;
Professor Hajime Kita, Institute for Liberal Arts and Sciences, Kyoto University 

This article describes how instructors designed and carry out the course "Practice of Basic Informatics." This course teaches the usage of information and communication technology (ICT) from the perspective that ICT is essential for intellectual production activities on campus. That's why it does not simply teach technology. It also instructs students on how to organize information with readers and audiences in mind and how to ensure that writing or compilations are logically consistent.

Associate Professor Sakai, Program-Specific Senior Lecturer Okamoto, and Professor Kita teach the Practice of Basic Informatics. They were interviewed about the concepts behind the course and what schemes they use to provide twice-weekly feedback to roughly 200 students.

The interview was conducted in February 2020 before the spread of COVID-19. The current situation of the course is briefly reported in the rounded-corner box at the bottom of this article.

Notes:
In response to the version-up of PandA and mksummary, we updated the image of how to download the assignment data on PandA and the demo video of how to use mksummary (June 7, 2021).

Profile
Hiroyuki Sakai: Associate Professor, Center for the Promotion of Excellence in Higher Education, Kyoto University. Ph.D. Started his academic career as Assistant Professor at the Graduate School of Science and Technology, Kobe University; then Assistant Professor at the Center for the Promotion of Excellence in Higher Education, Kyoto University. Took his current post in 2008. His field of expertise is educational technology. In addition to teaching and research activities, he serves as Managing Director for the production of Massive Open Online Courses (MOOCs), produced and published by Kyoto University on the global MOOCs platform, edX, and for the platform construction/maintenance and course production of KoALA, which is an on-campus version of MOOCs.

Masako Okamoto: Program-Specific Junior Associate Professor, Center for the Promotion of Excellence in Higher Education, Kyoto University. Ph.D. in informatics. Started her academic career as Program-Specific Assistant Professor, Center for the Promotion of Excellence in Higher Education, Kyoto University. Appointed to her current position in 2019. Her field of expertise is informational education. In addition to teaching and research activities, she served as Course Production Manager for the production of MOOCs and KoALA, together with Associate Professor Sakai. She has played significant role in the production of almost all currently available MOOCs and KoALA programs.

Hajime Kita: Professor, Institute for Liberal Arts and Sciences, Kyoto University. Ph.D. in engineering. Started his academic career as Assistant Professor at Kyoto University; then Associate Professor at the Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology; Professor at the Department of Research for University Evaluation, National Institution for Academic Degrees and University Evaluation; then Professor at the Academic Center for Computing and Media Studies, Kyoto University. Took his current post in 2013. His field of expertise cover a wide spectrum of topics including information engineering, system engineering, and educational technology. From 2013 through 2016, as Vice-Director of the Institute for Liberal Arts and Sciences, he took a leading role in the reform of liberal arts and sciences courses. From September 2016 to March 2021, he has been Director-general of the Institute for Information Management and Communication. He authored the textbook Programming Exercises: Python 2019 (corresponding page in the Research Information Repository KURENAI), which was well-received by Internet users due to its ease of understanding and has recorded 335,000 downloads till the end of March 2021.

Acquiring Essential ICT Skills Required for All Academic Activities in University

Could you tell me about the Practice of Basic Informatics course that you teach?

topics-28-photo01-7836

Associate Professor Sakai, Center for the Promotion of Excellence in Higher Education

Sakai/Okamoto: The liberal arts and sciences course, Practice of Basic Informatics, is intended to primarily teach first-year students, in an exercise format, the basic knowledge and skills in information and communication technology (ICT) essential for their campus life.

During one half of the year, consisting of 15 classes, the course covers topics such as basic computer operations, report or presentation material creation methods, information security and ethics, information retrieval methods, and basic programing.

There are other identically named courses, of which Sakai and Okamoto teach together a total of six classes, or more specifically, three classes each in the first and second semesters. The first-semester classes are for all students across all faculties, while the second-semester classes are basically for the first-year students of the Faculty of Agriculture.
Students taking the course for all faculties are primarily students in liberal arts/ social sciences/ humanities and number from 20 to 50 per class. Students taking the course for the Faculty of Agriculture number from 50 to 90 per class, because the course is a class-designated subject.

Professor Kita at the Institute for Liberal Arts and Sciences, who is with us here today, teaches one class together with Professor Hirohisa Hioki at the Graduate School of Human and Environmental Studies.

A class of 90 students per period is exceptional, considering that it is an exercise-oriented course.

topics-28-photo08-7919.png

Program-Specific Senior Lecturer Okamoto, Center for the Promotion of Excellence in Higher Education

The Faculty of Agriculture has a capacity of 300 students. To allow all of them to take the course, that class size is necessary. Given that three classes have a total of about 200 students, that means that about 70% of the first-year students in the Faculty of Agriculture join the classes, despite the fact that it is an optional subject.

This classroom has a capacity of 70. For a class of 90 students, this classroom and the one opposite are connected to simultaneously teach the class. A teaching assistant (TA) helps the students in the opposite classroom.

Could you tell me about how you teach in your class?

topics-28-photo02-7695.png

Associate Professor Sakai and Program-Specific Senior Lecturer Okamoto deliver their lectures in this classroom. Their lectures can be videotaped with the camera on the wall and relayed to the opposite classroom.

Of the 15 classes, the first two are allocated to lecturing on OSs and networks.

In principle, the subsequent classes are exercise-oriented. In these classes, we use the initial 10 to 15 minutes of the lesson hour to explain the content provided in the session. Subsequently, individual students do exercises using the textbook.
The instructor and TA interact with students when appropriate during the exercise hour. Besides it, we encourage the students to teach each other when they encounter an unclear point.

I see. Students are encouraged to teach each other.

Yes, they are. Different students require different amounts of time to complete the exercises due to the nature of the course content, which depends on the student's current knowledge and skills. For example, it depends on how familiar they are with personal computer operations; whether they have experience using Office applications, such as Word, Excel, and PowerPoint; and whether they are good at typing or not. Accordingly, we ask those who have completed their exercises faster than others to support other students nearby.

I have taken a look at the textbook. It contains a chapter on intellectual production. So you are also teaching topics other than how to use a PC and the Internet?

topics-28-photo04-chitekiseisan-small.png

Excerpted from the beginning of the chapter on intellectual production. It explains report creation and other similar activities as intellectual production (click the image to enlarge). You can download the text here (Research Information Repository KURENAI of Kyoto University)

The goal of this course includes helping students to acquire ICT skills essential for academic activities. I expect Professor Kita will go into more detail about this aspect later. To support abilities required for students' campus life, this course accordingly and impartially covers information-related abilities, such as library-based methods for information retrieval, ways of writing reports, and ways of making academic presentations.

For sessions dealing with library-based methods for information retrieval, we seek support from librarians, including Associate Professor Yumi Kitamura and Assistant Professor Chifumi Nishioka at Kyoto University Library.

The textbook is written primarily for Windows users and contains operations not always applicable to Mac machines. For Mac users, we made a separate Mac user-specific instruction manual. We give this to Mac user students and ask them to replace our explanations when appropriate.
We made the English textbook available here.

How do you grade students' work?

In every class, we give the students two assignments. They work on one during the class and do the other as homework for the day. The students submit these assignments using the assignment tool on PandA. We mark them and calculate the sum, after adjusting the weight of each assignment.

Do you provide any feedback on submitted assignments?

topics-28-photo03-7740.png

Associate Professor Sakai explains how his lectures are delivered.

The assignment tool on PandA has feedback/comment function. Using it, we provide comments to students along with their scores. Assignments include works like writing reports using Word, programming and so on. It is never a simple task to grade, and provide comments on assignments submitted by some 200 students by the next week's lecture. To make it even minimally easier, we use tools called "mksummary" and "placecomments", created by Professor Kita and Associate Professor Takaaki Aoki at the Institute for Liberal Arts and Sciences (currently working at Information and Communications, Nagoya University).

Use of mksummary and placecomments to Provide Feedback to 200 Students Biweekly

You receive practice assignments from 80 students in the first semester and from 200 in the second semester. It is a lot of work just to mark them. What kind of tool is mksummary?

The tool mksummary creates a list of answers submitted via the assignment tool on PandA. Using it, you can mark submitted answers successively.

Otherwise, when grading with the PandA assignment tool, you need to open one page for each individual student. The drawback of this system is that each time you finish grading one student, you need to open another page, which takes some work. In the case of reviewing quite a few students, you can be lost in tracking whom you have already marked and whom you have not; and it is not easy to review already input scores and comments.

As a solution to these drawbacks, mksummary was developed. It can be regarded as an auxiliary tool on PandA.

Could you show me how to use the tool in a concrete way?

topics-28-photo06-En-small.png

How to download assignment data, using mksummary on PandA (click the image to enlarge).

Yes. I will explain it using Windows. First, download the mksummary package from this website (KU members only) and open the "windows" folder. The folder contains the programs "mksummary.exe" and "placecomments.exe". You can download its manuals (Ja/En) from the link above.

Now, using the assignment tool on PandA, open the page that lists submitted materials. Click "Download All" at the upper right corner of the page. On the next page, check "All" and select "Excel format" for "Grade file" and download the assignment data (see the figure shown to the left).

topics-28-photo07-mksummaryR.png

Demo video of how to use mksummary (In Japanese, click on the image to start the player)

Drag and drop "grades.xls" from the download folder to "mksummary.exe" in the mksummary package. This lets the mksummary program run. When the program is completed, several files will appear in the assignment folder.

Double-click the summary.html file to open in your browser a list of submitted assignments on a single page, which allows you to view and mark them (see the GIF animation provided to the left).

Notes (June 7, 2021)
In response to the version-up of PandA and mksummary, we updated the image of how to download the assignment data on PandA and the demo video of how to use mksummary. The contents of the "windows" folder in the mksummary download package are referred to as "mksummary.exe" and "placecomments.exe" in the texts above, but the current version contains three files: "mksummary-for-p20-01.exe", "mksummary-for-p20-FB.exe", and "placecomments-for-p20-01.exe".

This is great. Am I able to view the attachments at the same time?

Yes, you can. Attachments are displayed as links. By clicking the link, you can view the linked file stored on your PC.

It is also possible to deduct points for late submissions uniformly. For actual grading, we first set a due date and the number of points for uniform deduction. After applying these settings, we mark the submission of each student in accordance with the grading criteria.

Are the grading criteria made known to students?

Yes, they are. When giving assignments, we also indicate the grading criteria. If a submitted assignment is unsatisfactory, we return the assignment, indicating which points are unsatisfactory in the comment field.
Although you can use PandA to provide comments, mksummary is convenient. Using it, you can view a list of assignments, which is useful for providing responses. You can consult your previous comments or copy and paste prepared text to respond to frequent errors. You can also check for any inconsistent application of the grading criteria.

I am impressed that you provide comments to each student carefully. How much time do you take for grading?

topics-28-photo09-feedback-small.png

A rendered image of delivering several classes in a week, while grading and providing feedback on assignments and homework for each class (courtesy of Associate Professor Sakai). The illustration depicts the first semester of academic year 2020. Classes are scheduled for Tuesdays and Thursdays.

The number of first-semester students differs from that of second-semester students, and different assignments require different amounts of time. So, I cannot say with certainty. However, report assignments received as Word files from second-semester classes take more than one day per week. The task requires careful reading of assignments submitted by 200 students. We sometimes even use a weekend to grade and comment on them.
Moreover, due to the nature of the course, we need to mark and give feedback on assignments worked on during class and those given as after-hours homework. Organizing this feedback is not a simple task, not only because of a large number of students, but also because classes are delivered concurrently to more than one class.

So, we need to proceed as efficiently as possible. In this regard, mksummary is truly useful.
The browser displays a bar to indicate your progress. Although it is a plain looking bar (laughs), we are grateful for it, as it helps the grader stay motivated by visualizing how many assignments have been graded and how many remain.

Now, could you tell me how to use the other tool, placecomments?

We use placecomments to reflect the marks and comments in the assignment data on PandA.

topics-28-photo05-2837.png

To do this, download the data from mksummary by choosing "Download in CSV format" and place it in the assignment folder. Then, drag and drop the "scores.csv" file into placecomments.exe. This lets the placecomments program run and create several new files in the source folder.

Then, on the original assignment page on PandA, select "Upload All" for the "grades.zip" file, and the results of your marking along with your comments are reflected in the PandA data. Meanwhile, during this process, select "Excel format" for the "Grade file" and check "Feedback comments," then "Upload."

Lastly, on the assignment page on PandA, press the "Release Grades" to allow students to view their grades and comments. (* For detailed operations, view this video created by Professor Kita.)

Thank you. The operation, however, seems a little complex for those who are new to it.

Although at first it may seem complex, it actually is not, and its substantial advantages become clear when you get used to the procedure. Moreover, you can consult the instruction manual and view an explanatory video (both of them are in Japanese).

I have heard that you use another special tool for your course.

Yes, the Classwork Assist Box. Professor Kita introduced this to me as well, and we've since been using it. It is a special attachment to Fuji Xerox multifunction machines. When a purpose-designed sheet of paper with a QR code is scanned, the Classwork Assist Box processes the student ID through optical character recognition (OCR: a function of recognizing text on paper and converting it to digital text). Furthermore, it automatically uploads data to PandA. On PandA, data from scanned sheets are uploaded in accordance with individual student IDs and look as if the assignments were submitted by each student.

There are few multifunction machines with a Classwork Assist Box function in Yoshida Campus. We use one installed on the first floor of the building where we deliver our classes (south building, Academic Center for Computing and Media Studies).

In what situations do you use it?

topics-28-photo10-7940.png

Peer review results on a Classwork Assist Box sheet, assessing each group's presentation with respect to individual perspectives. (This is a partially edited image. Click it to enlarge.)

We use it after group work and presentations, to scan assessment sheets produced by students through peer review. PandA certainly offers a similar function online. However, we purposely ask my students to write on paper because it is easier to make notes on the spot. We let students know in advance the perspective they will use to assess each other. They then conduct a peer review.

Scanned sheets are returned to the students with comments after the compilation of assessments for each group by a program written in Python. We give students some time during the class hour to read assessments of their presentations. They read them enthusiastically. Comments are largely sympathetic, probably because we tell them in advance that they will read comments in the classroom. You can see them smile while reading.

Course Developed in Backward Design and Its Effects

You are truly providing excellent follow-up services for students. Initially, what idea led you to develop the course Practice of Basic Informatics?

Sakai: The question seems most clearly answered by Professor Kita who took part in reorganizing and starting up the course. Could you answer this question, Professor Kita?

topics-28-photo16-7788.png

Professor Kita, Institute for Information Management and Communication. He formerly served as the Vice-Director of the Institute for Liberal Arts and Sciences at the time when Practice of Basic Informatics was reorganized.

Kita:The course is a result of unifying courses previously delivered separately at the faculty or department level. In 2016, when the Liberal Arts and Sciences curriculum was reviewed, course titles and credits were unified. Previously, in addition to "Practice of Basic Informatics", other course names were used, including "Computer Literacy and Practice." Moreover, a varying number of credits were granted for these courses. For example, the Faculty of Engineering set up its course as a single-credit course, while the Faculty of Integrated Human Studies set theirs up as a dual-credit course.

Along with the unification of the course title and the number of credits, the other question was what course content should be covered. We formed a working group within the Institute for Liberal Arts and Sciences and designed the subject matter through group discussions. At that point, we set three goals for the students: to acquire ICT skills essential for academic activities, to become independent ICT users, and to develop self-learning abilities to acquire ICT skills voluntarily and continuously. From this, we decided to write a common textbook.

Oh, that's this one, which Associate Professor Sakai showed me earlier.

topics-28-photo12-text-en.png

Textbook for Practice of Basic Informatics (Click the image to go to the download page on KURENAI.)

Yes. We wrote it with the help of Associate Professor Kitamura at Kyoto University Library and Professor Hioki at the Graduate School of Human and Environmental Studies. In 2017, Associate Professor Sakai also helped us in revising the textbook. In addition, many other individuals assisted us. Their names appear on the "0. Preface" page of the textbook.

As Associate Professor Sakai touched on a little, a high disparity tends to exist between students in the field of information literacy. We put a lot of effort in writing the textbook so that it could be understood by students who have never used a PC until their entry into university. The textbook is available with a CC license* in the Research Information Repository KURENAI of Kyoto University.

Meanwhile, the Faculty of Engineering and the Faculty of Science favor studying Linux and programing languages, so they use their own teaching material. Their lectures are delivered by the teaching staff of their respective faculties and departments.

* Creative Commons (CC) license: One of public copyright licenses, which is defined by Creative Commons, a nonprofit organization, and permits distribution of copyrighted works. For more information, visit this website.

The textbook shows a rubric at the end. Did you create the rubric by yourselves?

topics-28-photo13-rubric.png

Rubric used for Practice of Basic Informatics (Click here to view it in entirety.)

Yes, we did. When the content of the course was finalized, and prior to writing the textbook, we developed the rubric.
The course content is organized into topic categories, subcategories, and topics. We developed the assessment criteria (see the figure shown to the left) to assess the achievement of learners using a three-level rating scale. Level C is the level required to pass the course. Level A is a goal of this course. Level S shows the goal that learners should acquire in their whole undergraduate program.
The textbook was written for students to meet the assessment criteria shown by the rubric.

I see. You developed the textbook in backward design, that is, you moved from course goals to the course content. Could you tell me in what way you use the rubric?

topics-28-photo14-jikohyouka-small.png

Changes in self-grading based on individual perspectives on the rubric before and after receiving Practice of Basic Informatics lectures (courtesy of Associate Professor Sakai)

Sakai: I use it to tell students the concrete goals of the course and, at the same time, to help them look back at themselves and evaluate their progress before and after joining the course.

More specifically, I introduce students to the rubric in the first class. They get information about what they are going to learn and what achievements they are expected to make through the course. In about the 10th class, I use a questionnaire, the same used by Professors Kita and Hioki, to ask students to evaluate their position on rubric topics before and after the lectures.

Their responses, based on self-grading, indicate improvements roughly by one level (see the figure shown to the left).

The results reveal that, on the whole, students' self-grading results improved.

Kita: Yes. With individual topics, the number of Level A (a goal of this course) increased substantially before and after lectures.

topics-28-photo15-7824.png

Courses of this sort are likely to be misunderstood with regard to their purpose. People both on- and off-campus often criticize, "What is the point of a college teaching these topics when personal computer lessons are available in town?" or "Using a personal computer is supposed to be something students learn on their own."

This rubric is also meant to convince people with such views that this course responsibly trains students in rubric topics and allows them to advance as skilled second-year students.

For Students to Acquire Intellectual Production Skills Essential for Campus Life

Listening to your efforts, I'm truly impressed that the course is so carefully designed and delivered.

Thank you. As Associate Professor Sakai relayed, the main objective of this course is to teach the intellectual production skills essential for studying at a university. I hope that students learn the methods required to produce logically sound writing and to report data using standard style and wording.
I hope that they acquire the skill of devoting their energies to contemplating content, allowing the machine to do the tasks that the machine can do, such as assigning section numbers and inserting cross-references.

Such abilities will be essential throughout their campus life and after their graduation too. For this reason, we design and implement the course to allow students to receive proper training early.

Knowing more about the content, I will recommend the course to students I know.

Indeed, there are students who chose this course because their seniors told them that the course was good. These remarks are certainly encouraging.

That's nice. I don't think that in every course students get such careful instruction. Lastly, could you tell me about your future projects?

topics-28-photo17-7815.png

Currently, the course topics do not include the coordination of multiple machines via a cloud computing service. Including this in the future is in our thoughts. Some students ask questions, such as how they can link a smartphone to a personal computer, and what cloud computing tool is good for holding a group meeting. However, I do not commit myself to teaching this, as I am reluctant to force my students to create an account for such services.

Sakai: In some cases, among students working in groups, those with good knowledge of Office or Google tools introduce these to other students, and they all work using such tools.

Kita: Additionally, the number of videos that offer benefits for students are increasing. It is probably good that the course includes topics relating to video creation methods. We have also created videos as teaching materials, capturing us explaining how to use some of these tools. It is probably good to foster literacy in creating videos, not just for fanciness but because some things are more easily understood visually.

That said, the primary objective is surely to train students to be able to write properly.

I see. Thank you so much for your time today.



After the interview, the course delivery situation changed dramatically.
At Kyoto University, many courses went fully online. After going online, Associate Professor Sakai, Program-Specific Senior Lecturer Okamoto, and Professor Kita continued teaching Practice of Basic Informatics and requiring students to perform exercises for each lecture.
They say that the use of Zoom to deliver lectures offers an advantage in that videotaped lectures can immediately serve as video teaching material to thus facilitate the student learning by drawing on an unexpected stack of video resources.

Incidentally, individual instructors at Kyoto University reported on their delivery of online lectures in an event entitled "My Online Lecture@KU". The progress of Practice of Basic Informatics after going online was also reported at this event. Please use the link below to view the reports (KU members only).
"My Online Lecture@KU" #4, Professor KITA Hajime, Director of the Institute for Information Management and Communication + Associate Professor Sakai and Program-Specific Senior Lecturer Okamoto, Center for the Promotion of Excellence in Higher Education (May 22, 2020) (KU members only)

Contact us if you have any opinions or thoughts about this article.
We are waiting for information from informed people about educational improvements made using Kyoto University's internal ICT (information and communication technology). Please get in touch with us using the inquiry form.

Interviewers:Mana Taguchi and Takeo Suzuki
Article composition: Takeo Suzuki
Photography: Wataru Kono
Date of Interview:February 20, 2020

Published online: September 2, 2021 (Original article)
Updated: June 7, 2021 (Original article)
Published online: June 14, 2021 (English article)