Adaptive Learning and Comprehensive Assesment for students-A concept

Hi All,
I was thinking about this idea for some time now and thought of sharing with all of the experts.

The current state of K-12 education in India is in a flux. The national boards are making sweeping changes to the way students are assessed and graded in schools. However, all these changes fail to address some basic questions that are of vital importance to any student in K-12 curriculum.

The idea is to have an organization comprising of teachers and educators that would partner with schools in a city. The organization along with the teachers in the school would collect data related to each student’s assignments, tests and project work. All this data would be digitized and sent to the organization for a much detailed analyses. The analyses would be done for the following criteria:

1.Comprehensive Evaluation and Feedback: Evaluating students on a comprehensive basis is a pain staking and resource intensive task. And I think in India where Teacher to Student ratio in K-12 classes hovers around 50:1, comprehensive evaluation for students would remain an uphill task for schools. Also, problem with grading and marks in an assignment/test is that it does not addresses the basic question that where should a student focus on. Thus, students keep focusing on areas where they have scored less marks, but they fail to realize the real problem i.e. why they scored less marks. Also teachers don’t remember each students individual strength areas rather they would say a student is good in a particular subject. This detailed recall about strength and weaknesses would only happen if they could look into each and every problem in the answer sheets/assignments/project work. Due to the sheer volume of data it is difficult to give feedback to student on each problem of an assignment or a test.

The idea is to analyze the digitized data collected for each student and put it through a comprehensive evaluation. This evaluation in addition to normal grades and marks would also have a subjective aspect to it e.g. “this problem was conceptual and require your concepts in Laws of motion”, “this problem was tricky and requires the following set of tricks” etc. This data could then be collated across number of periods of assessment to give a feedback to students and parents about the broad skill areas of a student. Feedback could also be given to teachers about the skills, strengths and weaknesses of a particular student. So that there understanding about students learning is broad based rather than restricted to the marks he/she scores.

2.Adaptive Learning content: The content used in education across schools is prescribed text books, reference materials, internet etc. However all these mediums to distribute content to a student is not directed to what exactly a student require. The content is not related to student’s interest rather to some standard methodology of teaching a concept or a subject. If a concept is presented in a form that has direct applicability to a student in his life then understanding of the content greatly increases

The idea is to take collate the data from assessment, feedback and the areas of interest for a student i.e. music, sports etc. Wherever possible organize the concepts and subjects around these areas e.g. if s student plays some wind instrument then concepts of waves and sound could be taught by giving examples how he/she can observe those concepts while playing the instrument.

Also, all this data across in years in school could be combined to generate a student portfolio. The student have this prepared at the end of his/her K-12 education and could use it in college admissions and jobs

The idea presented here is in a very nascent form and would appreciate feedback from the community member about the idea.

Tags: Adaptive, Assesment, Comprehensive, India, Learning

0 members like this

▶ Reply to This

Replies to This Discussion

Permalink Reply by Ed Jones on October 13, 2009 at 5:56am

Abhishek, Welcome to CR20. And what a topic you have introduced!!

This is a pretty hard concept of operations to write down in a few paragraphs, so I'm sure you'll have trouble getting traction here. But, YES!!! this is the sort of direction we need to be heading.

My criticisms of public Education primarily center around the fact that we have made little progress indeed toward the sorts of goals you state. Some, I'm sure, may be very difficult, if not impossible.

And the current climate at least here in the US is not set up for such delicate innovation. Indeed! The news this morn includes a 1st grader sent to 45 days of reform school for carrying the swiss army knife he eats his breakfast with, That's the type of subtle, nuanced personal attention our public schools are currently capable of.

Yet lets give it a try. Start with a very simple thing:extraccurricular reading and learning.

Lets say we have a notably bright male, age 15. How many schools even have a mechanism to assure that someone recommends some good, appropriate books? Is he a visual/digital learner? Or would audio books be a better recommendation? Does he prefer action--ok most males do to some extent. But sports action or sci-fi action? Is his reading behind because of perhaps some issues earlier in school? What books would then be more appropriate? What has he read in the past that he can expand and grow upon? What else?

Abhishek, Do you by chance work in AI or Knowledge Management software solutions? What brings you to this idea? What else can we learn here?

Thanks!

▶ Reply

Permalink Reply by Rob Letcher - www.21stcenturyed.com on October 13, 2009 at 6:47am

The kind of personalization you speak of here was once possible, I think, when schools were smaller and is certainly one of the biggest reasons families homeschool. In a perfect version of this type of system, multiple lessons would be available in the "system" all written at varying reading levels. The "system" would present students with a lesson that matches their learning style, interest area, reading level, etc. while perfectly addressing the objectives intended.
The technology for this is pretty simple - not really AI, per se - just good sound logic with accurate tagging of content. The issue I see is from a content standpoint - imagine how many hundreds and thousands of alternative versions of lessons you might need to simply address a single set of objectives. Sure - there are a lot of resources out there already both free and for purchase, but then you need some method of storing and tagging them...and someone able to do both accurately.

▶ Reply

Permalink Reply by Abhishek Gupta on October 14, 2009 at 12:06am

Rob i think you have brought out a important issue of handling the operations part of the process. And its true that it is a challenge of having a content that caters to a wide variety of audience with different needs. Also, I agree to your point that tagging is the way to start of the process and AI is the way forward.

But to start of, we can have some very basic break down of interests of students. Lets say in India students are generally involved in Sport and Music. So start to of we can have courses, assignments and problems around Sports and Music. The content may not be exactly relevant to the sport or a insturment a particular student plays. But it would have atleast some relation with the student's interest. So if a student is interested in sports then content would have a definite theme of sports. I agree it is very difficult to have
content adaptable to all the sports that students generally play. But that kind of exhaustive content could be developed in due course of time. The moot question to ask is

Is this delta imporvement worth investing time and money?

There could be a private organization partnering with schools and parents, where teachers and educators come on board with this single vision and provide such kind of a service.

But the real question is whether parents and school would be interested to take such kind of service?

My views are that in India parents spend a awful lot on schools, tutions, books and specialized tests. However, there is no way data from these sources can be gathered at one place, analyzed and send to students, teachers and parents. And there is no indication of going forward on this front either in public or private domain.

▶ Reply

Permalink Reply by Abhishek Gupta on October 14, 2009 at 12:24am

Hi Ed...
I am currently into large scale ERP implementations. But the complications in this kind of work is similar, that is to figure out a disseminate right kind of information to people so that they are interested in it.

To start of the process we can maintain a detailed description of personality for a particular student. These details are primarily related to the content that a particular student would be interested in. I agree to your point that there could be a infinite possibilities of customization of the content based on students interest. But to start of we can have a broad break up of interests and then design the content around this. Lets take your example if a student is prefers action then we can start of by giving content related to sports and hope he/she would take interest!!But to be realistic we can build are content pool and start to cater to various other interests.

So if we are able to do this we are in principle talking about a private organzition that partners with schools and parents to provide such kind of service.

Also a important that you brought up was the past experiences of the student in different schools and during different periods of education. The service of collating the data to prepare a student portfolio caters exactly to that need. The data collected from assignments, project work and test could be summarized to have broad understanding of students learning.

But the question on which I would like your feedback is Even if we start small is this idea worth trying. And whether parents and schools would be interested to participate in such a activity?

▶ Reply

Permalink Reply by Ed Jones on October 14, 2009 at 11:28am

Abhishek,
Is it worth it? Depends on your feeling of commitment! People always want better for kids education. The question is at what cost to them in time and money.

"we are in principle talking about a private organzition that partners with schools and parents". Yes, probably. But what do you mean by 'schools'? Teachers? Curriculum directors? Superintendents, School Boards? State education departments? In the us, you might have to get good with different ones depending on what you want to put in the schools.

How about he National Council of Social Studies teachers group? Or, in the US we have a huge home school movement. Would you include them?

To me the fastest way to change the culture is to introduce something on the web that teachers will embrace (think Twitter, Ning, Wikispaces) as individuals. If you want to sell a product or service to schools directly, its a long, hard road.

Student portfolios: Here's a paper on the subject: The Student Data Backpack
(Margaret Raymond, p 145) One of the issues here is privacy concerns about schools collecting detailed stuff. Perhaps it would be more acceptable if an outside service was offered to parents?

Of course, kids are very changeable and hard to put into buckets for longer than a millisecond.

At this point I am thinking of eHarmony.com, the dating service, and their 29 Dimensions® of Compatibility. Is this the sort of software needed?

Or do we need something like that system that tells parents how to deal with difficult children, the key phrases etc that will put them back on track?

Lots and lots to think about!

▶ Reply

Permalink Reply by Abhishek Gupta on October 20, 2009 at 4:31am

Hi Ed,
First of all thank you so much for giving me such a helpful link about Student Bratpack, i went through the document in one sitting!!

The document is very comprehensive and talks about data capturing and assesment from the standpoint of policy makers, state goverment and fedral goverment. Therefore it captures the elements of data that are relevant for decision making for these stakeholders like financials, school administartive data etc etc.

My idea is to have a very sharp focus on assesment and learning related data generated by the students during classwork, homework, assignments and tests. Although the general debate about data gathering in field of education revolves aroung gathering data from Computer based learning("click trails") and computer based asssesments. According to the document about education data the idea of converting the text based data, that students usually generate, to a digital format is unviable as it involves multiple people, effort and finances. But I have a fundamental disconnect with such a idea. That is, everything that is on computer can only be captured but what is on paper is difficult to capture and analyze. In the near term this is almost impossible even in US and in India it would be ages that we would move to such a system. And actually I dont think we would ever move to such a system. Imagine the flexibility you have to solve a differential calculus problem on paper as compared to a computer. It is just one reason that learning entirely on computer can never be possible. Thus, it becomes mandatory to find some way to capture and convert data generated on paper and notebooks in a digital format for analysis.

For the time being if we assume that roadblocks related to privacy can be removed. Then a third part based "outsourcing" system to gather and analyze data can be developed . In fact this is one of the other strategy that could be applied from other sectors and industries that frequently use outsoucing for their less value adding operations.Teachers could be trained on a methodology of assesing answer sheets and assignments systematically. This methodology or the Standard Operating procedure needs to be carefully decided by experienced educators and it would lay the foundation of the "Operations" of the organization doing such a activity.

If you look more closely teachers around the world spend so much time in evaluating students.However, this evaluation has problems

-It is simplistic in nature i.e. based on some yardstick 1-10, grading, marks etc.
-As the number of assesments or the number of students increases this becomes a increasingly difficult task for teachers and schools to do. In India we have a additional problem around this that student regularly take practice test from a institution like Kaplan which test student for GRE and GMAT. Thus evalaution becomes a mundane, operational activity that teachers need to do as a routine job. Rather than giving a enriching feedback to the student for each question attempted.

I think atleast in India we can have a viable financial model of collecting fees from the user of the services and employing teachers to provide such services. As we can have good quality teachers who are trained to asses answer sheets in a cost effective way.