linklens

Visit Blog Website

3 posts · 1,977 views

Commentary on computers, programming and academic papers in the fields of Mobile Computers, Virtual Worlds, Second Language Learning, Cognition and anything else that happens to pique my interest.

Samuel Joseph
3 posts

Sort by: Latest Post, Most Popular

View by: Condensed, Full

  • July 2, 2009
  • 05:10 PM
  • 702 views

Sen et al (2007) Learning to Identify Beneficial Partners

by Samuel Joseph in linklens

Cited by 4 [ATGSATOP]So this is another paper in my attempt to finish the background reading for an invited paper in the AP2PC'07 workshop proceedings.  I believe I found this one following a citation trail from Ben-Ami and Shehory (2007) and I think I grabbed it because it had "learning" in the title.  Peer to Peer is mentioned in passing, but this paper is really about a multi-agent system where individual agents have learning capabilities.  I know the first author from a panel session in AP2PC'05, so that is another connection, but I can't really remember if I had some more complex motivation for printing out this particular paper last October.In principle I am reading this to help illuminate some of the ways that agent research can be of benefit to P2P researchers, but there is a part of me that is just interested in mathematical and algorithmic characterizations of "learning".  The paper itself introduces parallels between human and artificial agents trying to make critical choices about interaction partners; and this makes me think of the human interaction analogies in Ian Clarkes Masters thesis on Freenet, my own intuitions about pruning search in my NeuroGrid system, as well as the agent modelling in the paper I co-authored with Ben Tse and Raman Paranjape.  We are all humans and we interact with other humans most days, and so I guess it is no surprise that this sort of analogy crops up again and again; however I think there is a pitfall here.  Sometimes the analogies breakdown and our intuitions lead us astray - I think this is the case with mobile agents where our human experience of the greater efficiency of human face to face interaction suggests that sending a mobile agent across a network should be more efficient than static agents communicating with each other when in fact it is difficult to predict the relative efficiency of the two methods for mobile agents (Joseph & Kawamura, 2001).The goal of the authors research is to try and discover which learning schemes will sustain mutually beneficial partnerships between agents.  Apparently algorithms which achieve equilibria in repeated play have so far been restricted to two-player situations.  This paper examines a population of agents that learn through the reinforcement technique of Q-learning (Watkins & Dayan, 1992).  The authors restrict their system to one where agents search through repetitive personal interaction; not through referral.In the authors system each agent is of a particular type, and has preferences to interact with agents of other types.  Thus the potential reward that agents achieve through interacting with each other is a matrix of agents against types; and the matrix is designed such that some optimal solution of agent partnerships exists where no agent can get a greater reward by switching to interact with other agents.  Since the matrix of rewards is unknown to the agents, the Q-learning technique is used to update agents estimates of the rewards of interacting with each other.  Q-learning updates estimates through a combination of earlier experiences of reward with the current experience.  The extent to which experience influences current estimates is not varied, and the alpha parameter that determines this is not mentioned again in the paper, making a replication difficult.  However, in order to vary the agents level of explorative behaviour, i.e. the extent to which agents try out new interactions, the authors adjust the probability with which the agents select a random agent to interact with, rather than the one recommended by the Q-learning estimate.  By adjusting this probability over time exploratory behaviour is gradually reduced in what seems like a sort of simulated annealing.No particular justification is given for the particular parameter settings or the approach used, leading me to wonder what the basis for this approach is.  Are these techniques similar to others in the literature, or are they based on any empirical observations of real-world phenomena?  Nonetheless, initial simulation results in a static environment show that a slow decay of exploratory behaviour is associated with the system taking longer to achieve equilibrium, but also with a higher final average payoff for the agents.  This certainly makes intuitive sense.In subsequent simulations dynamic environments are explored where agents die off and are replaced if they fail to achieve a sufficiently high payoff within a certain timeframe.  As the environment becomes tougher and agents are killed off more quickly it takes longer and longer for the system to reach a stable equilibrium, although this can be mitigated by reducing the level of exploratory behaviour (see figure d is rate at which exploratory behaviour decays).  Again this makes intuitive sense.In further simulations we see that protecting young agents can also help the system achieve equilibrium sooner, which also makes intuitive sense, and makes me think of the use of karma in online communities; or at least the way that new users will be given an initial chunk of reward or karma points.  Not sure how strong the parallel is here, but I guess you could model an online community in terms of multi-agents looking for beneficial interactions.  New users are entering the community at a certain rate, and not hanging around indefinitely.  They will need to have positive interactions within a certain time period before they will effectively remove themselves from the community; which makes me think of that paper that shows the effect of existing social network patterns on incoming users (wasn't it something to do with closed triangular relations) - should re-read that for my thesis project, if I can find it (probably in disCourse somewhere).In a final section the authors experiment with introducing noise and we see that noise can have a similar effect to prolonging exploratory behaviour, i.e. taking longer to get to equilibrium, but perhaps finding a higher optimum.  My main concern with all this is the relationship to the real world where systems may spend much of their time away form equilibrium.  I see connections to other work that I have done on the evolution of intelligence (where we compared our models with populations of animals in the real world) and online communities, but a lot of the modeling decisions seem to be soewhat arbitrary.  It would be nice to know what was motivating them.My references:Joseph S. & Kawamura T. (2001) Why Autonomy Makes the Agent. In Agent Engineering, Eds. Liu, J, Zhong, N, Tang, Y.Y. and Wang P. World, Scientific Publishing.Sandip Sen, Anil Gursel, & Stephane Airiau (2007). Learning to identify beneficial partners Working Notes of the Adaptive and Learning Agents Workshop at AAMAS... Read more »

Sandip Sen, Anil Gursel, & Stephane Airiau. (2007) Learning to identify beneficial partners. Working Notes of the Adaptive and Learning Agents Workshop at AAMAS.

  • June 26, 2009
  • 09:36 PM
  • 658 views

Kim (2008) The Role of Task-Induced Involvement and Learner Proficiency in L2 Vocabulary Acquisition

by Samuel Joseph in linklens

Cited by 1 [ATGSATOP]Another paper that I am reading as part of a meta-analysis of second language vocabulary learning. I had started to read this and then paused for three weeks while I read three background theoretical papers (Laufer & Hulstijn, 2001; Hulstijn, 2001; Hulstijin, 2003) that made this one much easier to understand.This paper is an experimental study in two parts designed to test L&H's involvement load hypothesis. One concern is control of time on task, since this varied in L&H's experimental attempt to assess involvement load hypothesis. Knight (1994) apparently brings this issue up in general for things like dictionary look up tasks. All through I was concerned with precisely how vocabulary knowledge was being measured. Like Folse (2006) Kim used the Vocabulary Knowledge Scale (VKS; Paribahkt & Wesche, 1993) but I still wonder what L&H used - later on it is described as providing L1 translation or English explanations. Laufer's (2003) experiment gave support for different performance based on different levels of involvement load, however another experiment in the set gave varying performance for three tasks that were supposed to have the same involvement load (distribution was different?). Am keen to know Laufer's explanation of that - that paper also on our reading list?Laufer (2001) apparently indicates that involvement load construct should generalise from textual to face to face audio situations, which I had assumed, but good to be able to reference that assertion given the wide range of studies we are applying the concept to.  I was unsure of the meaning of interactionally modified input versus interactionaly modified output, and in particular the concept of premodified input, although this is in the context of L&H(2001) that I guess I should be reading.I was concerned about the random assignment implications of the split between the two experiments. One of the experimental groups from the first experiment is compared with a group constructed for the second experiment, which I think was run subsequently, and although similar had a slightly different mix of ages and nationalities.Another concern is that it seems we could explain results independently of involvement load. In the reading condition the learners attention is only drawn to the target words through emphasis and glossing. In the gap-fill condition the learners attention is drawn to 15 words, and in the composition and sentence writing conditions the learners attention is drawn to the 10 words they will be tested on. Purely in terms of attention one might expect to see the results that were achieved. In the experiment that tested the three different involvement load levels, the immediate post test only distinguished the composition group as significantly higher, while the delayed post test distinguished all three - there was no interaction or main effect for proficiency level. The second experiment made no distinction between the composition and sentence-writing tasks. I had been wondering earlier if the results could all be explained in terms of receptive/productive or active/passive differences, although the significant difference between reading and gap-fill at post-test could not, but now I realise that there were 15 words being brought to attention in the gap-fill task, it seems that the results can all be explained in terms of attentional resources. Another question is whether the comprehension questions needed understanding of the target words in order to be answered (looking at appendix b I would say not really).I am concerned about the bias of using the VKS tests, and the author expresses some concerns as well. I find the alleged pedagogical implications sit uneasily with me, since I am not sure that showing a benefit on a VKS test necessarily indicates that the learner has gained something of importance.  The key problem here is that the VKS sentence generation task could represent various sorts of ability on the part of the learner, e.g. that they memorized a sentence containing the word versus actually generating a novel sentence.  In particular it seems that if a learner was specifically practicing sentence generation or doing essay composition for a particular set of vocabulary that this would increase performance on the test through a practice effect.  It seems to be obvious that practicing a productive skill would lead to higher performance on productive tests, whereas practicing a receptive skill would lead to benefits on receptive tests.  The question I would like to know the answer to is what kind of transfer do we get cross-task, and thus motivational concerns aside, what is the most efficient approach to take to maximise ability on both receptive and productive tasks.Reading proofs of our soon to be publshed paper on vocabulary study (Joseph et al. 2009) I am struck that as we discuss how to make tests more and more challenging, we are not addressing the goal of the language learner. We are arguing that gradually more challenging tasks maintains motivation and boosts long term retention, but the real question should be what is the long term task that the learner wants to succeed at. Clearly looking up a word in a dictionary can help a learner understand a sentence they are reading. The question is then whether other activity related to that word should be undertaken. The usual argument in L2 is that if nothing else is done then exposure to low frequency words will be insufficient for the learner to avoid having to look the word up again in future. I guess the real question is whether some sort of "artificial" re-exposure to the word will be a more efficient way of increasing the likelihood of future sentence comprehension, versus using that same time to just do more reading ... and what kind of experiment could actually test which approach was more efficient? I guess one could have learners perform a reading comprehension task, and then have one group perform another reading comprehension task, while a second group did vocabulary review, and then both groups would be tested on another reading comprehension task that was of comparable level and contained similar words. So for this kind of experiment we would need three different texts of comparable length, involving the same "target" vocabulary?Depending on the results of such an experiment an argument could be made to say that although explicit vocabulary study was not recommended, that selection of subsequent texts for additional comprehension practice could be selected based on which words were looked up by a learner, in order to increase the chances of a rewarding experience - which is linked to overall motivation issue, i.e. should the learner be reading anything other than texts they specifically select themselves?[A great deal of research has shown that when learners study definitions alone their ability to comprehend text containing the target words does not improve (Graves, 1986; Stahl & Fairbanks, 1986)] from Joseph et al. (2009), so I wonder if doing essay composition, or gap filling leads to improvements in text comprehension.[N.B. The Kim paper also references some more studies showing the importance of negotiation that I was previously associating with Newton (1995), i.e. de la Fuente (2002) and Joe (1995, 1998) although latter focused on generative rather than negotiated tasks?]Kim, Y. (2008). The Role of Task-Induced Involvement and Learner Proficiency in L2 Vocabulary Acquisition Language Learning, 58 (2), 285-325 DOI: 10.1111/j.1467-9922.2008.00442.xMy ReferencesJoseph S.R.H., Watanabe Y., Shiung Y.-J., Choi B. & Robbins C. (2009) Key Aspects of Computer Assisted Vocabulary Learning (CAVL): Combined Effects of Media, Sequencing and Task Type. Research and Practice in Technology Enhanced Learning. 4(2) 1-36.  Kim's ReferencesArlov, P. (2000). Wordsmith: A guide to college writing (Cited by 3). Upper Saddler River, NJ: Prentice Hall.Barcroft, J. (2002).... Read more »

  • June 18, 2009
  • 05:47 PM
  • 617 views

Csikszentmihalyi & Hermanson (1999) Intrinsic Motivation in Museums: Why Does One Want to Learn?

by Samuel Joseph in linklens

This is another paper that was recommended to me by Peter Leong who is teaching a course in Second Life this summer for the College of Education at the University of Hawaii. We are trying to better understand how we might build engaging learning spaces in Second Life.Reading this paper I started wondering what proportion of the population went to museums. Superficially I imagine computer games and films/tv to be far more frequently consumed by the general population, although since having children I realise what a valuable resource museums are. Is going to the cinema more popular than going to the museum? I guess the big difference is whether you are asking your audience to sit in a chair or walk around, and whether they are hoping for thrills rather than to be made to think. One imagines that theme parks are more popular than museums, but again it would interesting to know the real statistics.Csikzentmihalyi's concept of flow was mentioned in the McClelland (2000) paper I blogged about previously. Although it seems like Pine and Gilmore's experience realms diagram is a subdivision of flow, at least since reading Csikzentmihalyi's paper he mentions flow in the context of watching a basketball game, so the implication is that one can get sucked in to a state of flow for both passive and interactive experiences, and either absorptive or immersive experiences? However I am less clear about this latter dimension, I guess immersion is where you are totally immersed actively in a role, or in a passive appreciation of something. Funny as I would call that being absorbed, but absorption for P&G seems to be more about maintaining a distance from the thing you are observing, e.g. for an educational experience where you try and work out how something works.Csikszentmihalyi & Hermanson (1999) distinguish extrinsic and intrinsic motivations. They argue that museums must rely on intrinsic methods of motivation. A good part of the paper is taken up describing the flow concept, and I the range of activities the authors suggest can induce flow are wider than I expected. Flow apparently relies on activities that have clear goals and appropriate rules:Conflicting goals or unclear expectations divert our attention from the task at handHowever this is at odds with some of my experience programming, where I am changing goals as I explore different possibiilties. It seems to me there can also be a state of flow as I explore and reject possible goals, although I take the main point that this does require some shift in level of attention. Given clear specifications and little ambiguity one can become completely immersed in programming to specification, but sometimes the most interesting solutions come from questioning the goals and expectations to find alternative solutions to the real underlying problem.The authors also mention that as skills increase, the challenges of the activity must increase to maintain flow, which reminds me of the suggestion in other motivations literature (Dornyei?) that tasks should be just hard enough and not too hard to maximize motivation, although I have yet to find any empirical studies which back this up. The authors also cite a number of references to support the assertion that affective processes can be as important as cognitive processes in learning, and this ties in to ideas about memory being strongest when things are linked to emotionally charged events. Further discussion of flow includes the assertion that:when involved in the activity, the individual fully expresses the selfalthough sometimes when I am in a programming or writing flow I think I lose my "self". Overall the discussion of flow is interesting, but it only seems to tie weakly to the design challenge of museum exhibits. At least it is not clear whether the state of flow which I associate with much more focused activities is necessarily the state we should be trying to induce in museum visitors; although arguably it would be no bad thing to have visitors losing themselves in the exhibits. The authors provide the diagram shown above to indicate one approach to structuring exhibits or experiences at museums. There is the "hook" that piques initial interest, opportunities for involvement and then a set up for intrinsic rewards that hopes to stimulate flow. There are many good suggestions such as trying to connect exhibits to the individual visitor and presenting things as perspectives rather than fact:Information that is presented as true without alternative perspectives discourages the motivation to explore and learn moreAlthough this is slightly ironic as the statement itself is one of fact rather than perspective. There are further discussions of the conditions for flow, such as the suggestion that displays should provide information by which visitors can compare their responses to other standard(s?), and that supportative environments provide people with choices, and acknowledge their perspectives or feelings; however I find these difficult to conceptualize without more concrete examples. However the authors do acknowledge that as yet there is:no table where we can look up the elements that will attract the curiosity of difference types of visitors.Just started having this idea about doing roleplays and paying SL residents to come in and be actors for some relatively low rate of Linden dollars. Arguably they would be more fun to interact with than scripted bots, would allow for the possibility of "unscripted" emotional interaction. Although of course the payment aspect might negatively effect the social interaction. What about tasks or role plays where you don't get paid unless a team works together. An extrinsic reward may undermine intrinsic motivation according to this paper, but museum's pay actors to form living exhibits in their museums ... I wonder if one could create something engaging ... could the activity be interesting enough that people would get involved anyway? I wonder what kind of games have been built in second life already? I don't hear much about that, but then I haven't looked. It seems like you could have a murder mystery, or ecological mystery - could have a subterranean level with dwarves or gnomes (or menehune) suffering from a disease and rather than just find a cure, you have to convince the dwarves to change their behaviour which requires talking to multiple NPCs and working out a convincing argument - all much easier to make believable if you can make the dwarves real people playing roles ... they can judge themselves whether they have been convinced by what the team of detectives come up with, and be in a position to hand out prizes (in a suitable story context) when they feel they have been convinced. Of course we also have to deal with new people popping in at different times, but could deal with that by restricting the number of entrances; could have a leader board of time to solve the problem - I guess the whole thing could reset after a certain time - but what we really want is to allow multiple actors to come in and have their teamwork be required to solve the problem - I guess there could be multiple versions of the game - one if there is only one avatar and other harder versions if there are more avatars around ...Some related SL activity includes Second Life Singers, and also a Macbeth interactive experience, Virtual Hallucinations, and the OSU Medicine Testis Tour.Also an interesting blog on Education in Second Life.Mihaly Csikszentmihalyi, & Kim Hermanson (1999). Intrinsic Motivation in Museums: Why Does One Want to Learn? The educational role of the muse... Read more »

Mihaly Csikszentmihalyi, & Kim Hermanson. (1999) Intrinsic Motivation in Museums: Why Does One Want to Learn?. The educational role of the museum By Eilean Hooper-Greenhill, 146-160. DOI: http://books.google.com/books?hl  

join us!

Do you write about peer-reviewed research in your blog? Use ResearchBlogging.org to make it easy for your readers — and others from around the world — to find your serious posts about academic research.

If you don't have a blog, you can still use our site to learn about fascinating developments in cutting-edge research from around the world.

Register Now

Research Blogging is powered by SMG Technology.

To learn more, visit seedmediagroup.com.