jim.shamlin.com

Evaluating the Effectiveness of Social Visualization Within Virtual Communities

This article presents the results of an observational study about "social visualization" in the context of Cyber Mentor, a virtual community for German schoolgirls who have an interest in science, mathematics, and technology. (EN: As such, there's a lot of metadiscourse about the study that I will not annotate in great detail).

BACKGROUND

The concept of "virtual community" is defined in the standard way (regular interaction among a body of users on a specific topic of interest in a specific virtual location).

The "technology acceptance model" is described, presumably to address concerns that a study that involves technology may be skewed by the participants' attitudes toward technology itself rather than the intended subject of the study.

The term "social visualization" pertains to the method by which information about a member of a community - including their characteristics and relationships to other members of the group - can be illustrated, how other information may be detected or inferred from this model, and how the behavior of others ins influenced by this perception.

A few different models for visualization are noted: the presentation of a user as a "dot" that is connected by lines to other users to represent individual contact or grouped in a circle with other users who belong to a given group, graphics to represent the user's length of membership and frequency of participation, and the hierarchical representation of a discussion forum as a collection of messages that belong to a thread that is a branch from a subtopic that is a branch from a higher-level topic, each associated to the person who has posted.

The author is careful to note that the behavior the study attempted to observe was behavioral, and concerned itself with the actions taken by a user rather than the particular details of their personal identity, which would bring about issues of privacy, especially given the study was done of young girls (a group that people feel unusually protective of) in Eurpoe (a society that is unusually protective of its citizens in general).

(EN: While a sensitive topic, evaluating the way in which demographic characteristics coincide with behavior may be an area of interest, just not the topic of the present study.)

AIM OF RESEARCH

Social visualization tools aim to present information about the members of their community in order to gauge the level of activity and discover ways to stimulate interaction. Various tools exist for doing this, and the aim of the study was to evaluate their effectiveness.

The author concedes some qualities of social interaction (for example, the brief period of fascination in which a new member of a community is temporarily hyperactive before settling into a less active long-term state) and the methods used to avoid or account for the influence of these phenomena.

Also, "activity" is defined as being a combination of behaviors: the number of logins, the number of posts to forums, the number of "chat" messages, and the number of private messages sent within the community. The presumption is that it is in the interest of a community to have a high level of activity among its members.

METHOD

The "cyber mentor" community was built "from scratch" for this research project, capitalizing on interest in Germany to foster girls' interest in technical professions. It was not clear how this was promoted, though given the duiration of the test, I suspect it was done with the cooperation of the schools. Also, because of protectionism, they needed to get parental permission to allow girls to have access, which also helped to ensure that participants were genuine.

The site was built using open-source technologies for discussion boards, chat rooms, and personal messaging among members, with some custom development to stitch these components together and associate behavior with user profiles.

The visualization tool used was "cyber circle," which indicates the individual's level of participation and gauges their level of activity against that f other users. Visually, this is depicted as a "dart board" layout with the user at the center, members of groups at the edges, and a "dot" between them to gauge the flow of information (which of the two was more proactive in communicating with the other).

SUBJECTS AND STUDY DESIGN

The author documents his subjects (231 females, grades 6 to 13, median age of about 15).

The duration of the study was September through June (presumably coinciding with a school year). Of interest: the author defines the first two months as a "starting phase," the third as a "consolidation phase," months 4 and 5 as a "short-term phase", and 6 through 10 as the "long term phase," noting the implementation of the visualization tool began after the third month.

The study involved the collection of data: the count of logins, posts, personal messages, and chat messages

RESULTS

The notion that activity in a virtual community decreases after a starting phase was confirmed by the data collected.

The notion that the addition of a visualization tool would increase activity during the "short term" seemed to be confirmed by an increase in the amount of interaction among participants (posts, messages, chat lines) but did not result in a corresponding increase in the number of logins - i.e., there was more activity in each session, but not more sessions - in the two months following the introduction of the feature.

While there was a decrease between the short-term and long-term levels of activities, comparing the long-term activity to the period before the introduction of the visualization feature shows a slight increase in the activity in discussion forums and chat rooms, but no significant difference in the number of logins or private messages.

LIMITATIONS AND DIRECTIONS FOR FUTURE RESEARCH

The author concedes the absence of a control group (a second community in which a visualization feature was withheld), so the study cannot discount the possibility that the results observed were due to other factors. Given that there were only 231 participants, they feared that splitting them into two groups would result in communities that were to small to remain effective.

Another limitation was that the study was purely observational and anonymous: no attempt was made to gather information from participants that might explain their behavior or to subdivide the body of participants into groups whose behavior was characteristic and distinct: they were all lumped together.

Finally, given that the community was very small, the findings may have been affected by the size of the group. Behavior might be different in a large community, and it's conceded that the user interface of the "cyber circle" tool (a dartboard layout) may not be feasible in a community where a single individual has more than a few dozen contacts.

CONCLUSION

Based on the study, the conclusion is that a visualization tool can have a positive effect on the level of participation within a small virtual community, but more research is needed.

(EN: based on the nature of the study, the small group of subjects with distinct characteristics, I'd be reluctant to use this as a basis for making assumptions of a larger and more diverse group of people, who participate in a larger community over a longer period of time. Given the lack of a control group, the causal correlation is very weak even for the limited group to which it may be applicable.)