On Search and Discovery: The ADL Object Registry and Repository Infrastructure
Eric J. Roberts Philip V.W. Dodds
Over the past 10 years, the Advanced Distributed Learning (ADL) Initiative’s Sharable Content Object Reference Model (SCORM) has evolved to provide a modular, object-based basis for instructional subject-matter content. SCORM has solved key interoperability issues across many learning systems in industry and government and has received wide-spread international adoption, becoming a de facto standard in many communities of practice. While SCORM advances the state of the art in the design and creation of interoperable and reusable learning content, it does little to address the problems of finding and re-using content once it has been created.
Is this a real problem? Is this a problem that ADL must address? There are lots of libraries, after all. There is a huge volume of instructional content available on-line. A simple search quickly identified IIEP-UNESCO, OER Commons, Education Network Australia, MERLOT, OpenCourseWare Finder, and the National Council for OnLine Learning. All of these organizations provide massive repositories of information, much of it refereed.
The Nature of the Problem
It turns out that searching for content is a problem that ADL must address. Part of this necessity derives from SCORM’s success. In July of 2006, the Department of Defense mandated that any DoD entity seeking to acquire new web-based content must consider making that content SCORM-conformant. This action was taken to reduce costs, avoid re-creation of existing learning materials, and to enhance their interoperability and reusability. This mandate also requires a search of existing repositories of content to see if equivalent material already exists before any acquisition can proceed. This latter requirement seems a completely reasonable way to reduce costs and avoid re-inventing the wheel.
Unfortunately, there is no organized way to do that.
There is no way to search all of the different content repositories in the DoD. We don't know where they all are or even how many currently exist. Furthermore, there already are systems in place called DAVIS / DITIS (Defense Automated Visual Information System / Defense Instructional Technology Information System) that can be searched for instructional content. DoD entities are supposed to register their content in those systems, but the mandate to do so is not enforced, and developers frequently choose not to make the effort. In order to escape the same fate, ADL must find a way to make compliance easy and worthwhile.
This need is compounded in a couple of interesting ways.
Multiple communities of content creators and users exist with multiple content repositories and with few means for bridging and reusing learning objects among, or even within, the communities.
Figure 1—Isolated communities of content creators
Further, compared to publishing and library science practice, learning content is difficult to manage. The best instruction is situated in a unique context that is tailored to the needs and objectives of individual learners. Given ADL’s advocacy for a granular approach to the development of instruction, that means that context typically should be stored independently of content. So a lack of common practices for creating, storing, and describing learning content -- and contexts -- makes it hard to organize material on any large scale and to solve the Search problem.
Yet, making content visible and available more broadly should increase the return on investment to develop it.
And, indeed, that is what happens.
Chrysler Academy, for example, trains 180,000 members of the carmaker's sales force, staffing 4,000 independent dealerships. They use a blended approach to learning, including live courses and events, performance support, online virtual classrooms (for synchronous instruction) and self-paced, web-based courseware (for asynchronous instruction). Using SCORM as a standard for the latter, they have realized savings in development costs of 8-10 percent and in maintenance costs of 80 percent. At a recent conference, they showed how one self-contained instructional object can appear in five different courses and how such a granular approach to building content supports course delivery in multiple languages and increases the pool of developers and vendors available to build customized content. Interoperability of content independent of context and mindful reuse of content in user-friendly repositories together make these capabilities possible.
Unfortunately, achieving these goals may not be rewarded by the Department of Defense. Indeed, if a DoD course developing organization realizes efficiencies and significant cost avoidances, its next year's budget and personnel allocations may reflect that – by being cut. Chrysler's incentives do not align with those of the DoD.
In Future Shock, Alvin Toffler showed that it is possible to order as many as 10,000,000 unique Ford Mustangs from a catalog (by mixing and matching engine, and radio, and upholstery, and paint combinations). Ford allowed all these options because people prefer to have it their own way. We mostly prefer choosing a custom-tailored wardrobe instead of shopping for clothes off the rack, and, for instruction, having developers work to satisfy our personal requirements instead of learning from one-size-fits all courses. Searching existing repositories of learning objects is unlikely to provide exactly what we want / need, but it generally saves development costs, is more satisfying, and can produce much more effective instruction.
Searching is difficult when the structure of the repository doesn't readily reveal itself – as a traditional, physical library does. Nevertheless, the logic of re-using and re-purposing content remains compelling. One of the biggest challenges for instructional product developers is establishing the requirements for what it takes to know something and be competent in performing particular tasks. This invariably leads to heated and time-consuming disagreements among subject-matter experts – especially when softer skills such as those required for non-kinetic warfare, which involves practical communication skills, knowledge of culture, local economics, and the like, are involved. Using proven, stable, vetted content is a way to meet this challenge.
One of the original benefits of technology-based instruction is protection against curriculum drift. In instructor-led classrooms, the content is a function of what the instructor chooses to emphasize – often "war stories." In technology-based programs, the same information is available to all learners. Shorter development times and stable content mean learners get what they need more quickly. The granular approach ADL and others recommend also increases the likelihood that learners will get only what they need, with increased individualization of instruction, reduced seat time, increased achievement, and earlier availability for duty station assignment. Product developers who can re-use proven content also can devote increased attention to context, again increasing the individualized nature of the experience for the learner.
The logic holds.
What is needed, then, is a simple means for content developers and managers to make the digital content they develop visible and accessible to their primary users as well as new audiences so that some form of return on investment is possible. That is the goal of the ADL Repository (ADL-R). ADL-R seeks to provide an easy means for locating existing content that can be re-used or re-purposed in multiple applications inside the DoD. In short it provides global visibility for objects while retaining and ensuring local control over access to them.
The Requirements of a Solution
Here is how it could work:
Imagine that electronics technicians are learning to troubleshoot faults in an unfamiliar avionics subsystem, and that they have been certified to work on similar systems. A search for relevant content would want to know: a) what make, model, and version of avionics subsystem the technician is dealing with; b) what skills are needed to repair it; c) what skills the technician has mastered; and d) what procedures and skills are needed for the specific subsystem at hand.
This scenario assumes that:
- database exists with the exact configuration of each avionics system.
- Someone has defined a skills taxonomy for the system under consideration.
- A profile exists of the technician’s proficiency.
- Instructional content exists for this system.
- Instructional strategies exist to prepare this technician to troubleshoot this system.
Assuming this information exists and is accessible, one can imagine the development of a service or an ‘agent’ that can identify and retrieve instructional content that is necessary and appropriate for the technician’s needs from these sources.
Context provides the criteria required for discovery. It may be simple or complex, it may be automated, semi-automated, or defined manually.
Ideally the discovery of content objects might involve a process like:
- Develop search criteria from the local context.
- Go to a master index of relevant repositories.
- Discover what relevant content is available within the repositories.
- Determine if the repository access rules allow the content to be retrieved.
The problems with this approach are that no common practice exists for defining criteria for learning material, a master index of repositories has not existed until now, and the ability to search individual repositories and access their content is limited if it exists at all. To support this approach, a new architecture is needed. ADL has developed the ADL-Registry to meet this need.
In ADL-R, a registry of repositories provides a single place to go to find out where learning objects may be found. In such a registry, its index, ‘metadata’ describing the content of objects, and other object packaging information could be searched directly or mined by a discovery service. Publishers of learning objects who wanted their content to be found could register their objects and provide information about their content. Repository and content developers could establish and enforce rules for accessing and using content.
The ADL-R approach is simpler than the management and federation of library collections, and therefore more scaleable, because only information about the content is centralized—not the content itself. ADL-R also increases the precision of searching both by enabling sophisticated search services and by narrowing the scope to intentionally published objects. It does not attempt to add value to distributed collections by introducing new search algorithms. Instead it provides an infrastructure for organizing content metadata found in distributed repositories in a structured and predictable fashion. ADL-R is developing basic search services for using this metadata organization and for opening up the metadata to other search mechanisms, through special arrangements with web crawlers such as Google.
The emphasis on a registry versus a repository allows individual organizations to control their own content. They determine who is allowed to access the content in the same way they had in the past. They surrender nothing but information about what is in their repositories.
As in all previous ADL initiatives, the development of this registry is intended for the benefit of DoD and affiliated communities of practice. And, as in all ADL initiatives, it is freely available for use by all other communities of practice interested in solving similar problems.
How the Architecture Could Work -- Differently
Discovering a content object and knowing where and how to obtain the object are different matters. Currently we rely on Uniform Resource Locators (URLs) for locating content on the Internet. URLs have been highly successful tools, but they tie resources to their current network, owners, locations, and local file paths, which are included in the URL. When the resource is moved, the connection is severed and the URL no longer works.
ADL-R links, or ‘resolves’, an identifier to a variety of information, services, and functions that manage the use of an object. Linking an identifier to a location is only one of the services enabled using this approach. For example, authentication services could be located to determine who may access specific content or repositories. Other business rules might be located that protect intellectual property rights. Life cycle management policies and services could also be found.
The tool underlying ADL-R is called the Handle System. It was developed by the Corporation for National Research Initiatives in the mid 1990s, and has seen wide adoptions since that time. The Handle System defines globally unique identifiers that can be associated with various types of information about an object. The system defines how to build a registry of handles and the services needed to locate information associated with a handle. In the Handle System, a unique name, or handle, is created for each object and stored in a handle server along with a pointer to the object’s location. The location of the object is obtained by a resolution service that asks the handle server for the location information, among other things, much as the Domain Name System today resolves Web addresses to specific Internet Protocol addresses when used in a browser.
Why This is a Good Approach
Many approaches to repository and content management emphasize one of the three elements -- context, discovery (identification), or resolution (location) -- over the others. ADL-R treats all three equally and in parallel. Also, efforts to federate repositories often end up over-defining access protocols, metadata sets, and complex services. ADL-R is to remain on a level above most of the infrastructure and concentrate on means to expose services, data, and capabilities so that access and use can be negotiated rather than remaining fixed and predefined. Communities of practice can then develop independently of one another as required, but still be able to create bridges of interoperability.
Problems Remaining
The discussion above explains the intent and logic of ADL-R as do others (e.g., Dodds & Fletcher, 2004; Fletcher, Tobias, & Wisher, 2007). However, it doesn't address the most fundamental problem in Search: the Hermeneutic Circle (e.g., Dilley, 1999). This problem emphasizes that in order to learn something, it is first necessary to know something about it. The classic illustration is using a dictionary. If you want to check the spelling of a word, you first have to have an idea about how to spell the word.
Another illustration concerns textual decomposition, which, like other forms of information processing, is an intensely busy cognitive activity. In addition to decoding the characters displayed on paper, computer screen, or wherever, the learner must also bring to bear masses of public knowledge, or ‘intertextuality’, to continuously interpret and assess the message. If you are completely new to a domain, you cannot learn it.
That's a problem that ADL-R does not address. So...
What We Really Would Like to Have
ADL-R properly evokes serendipitous searches in bricks-and-mortar libraries. What we ultimately want is what the physical library permitted. You would go to the card catalogue to find the call number of the book you needed. On going to the stacks, you might be annoyed to find that the title you wanted was checked out or shelved improperly. The adjacent titles, however, on both sides of where your book should have been, treat the same or similar subject-matter. Looking through the Tables of Contents and the Indexes at the back of these adjacent titles, you often would find that the subject you sought to address could be treated differently – but equally well – using these other references.
This is an elegant system.
There is an even-better possibility, however. Here is a personal anecdote of how it can happen.
Searching for references at the Eisenhower Library at Johns Hopkins University to an obscure educational researcher, I was having no luck. Spotting a friendly-looking guy who I thought I recognized as Julian Stanley, famed psychometrician and founder of the Center for the Study of Mentally Precocious Youth, I asked him if he had any knowledge of the researcher I was trying to find.
“Oh, yes, of course. I wrote about her in, uhm, this book right here,” he said pulling one of his own titles from the stack. “Here it is. Here is the reference. I think you’ll find her just over there.”
So that is what I want: a system that will help me find what I want when I don't know what I want.
That's when the problem of Search will be solved. Clearly, not an easy task, but an important one.
References
Dilley, R. (ed.) (1999) The Problem of Context. New York, NY: Berghahn Books.
Dodds, P. V. W., & Fletcher, J. D. (2004) Opportunities for new “smart” learning environments enabled by next generation web capabilities. Journal of Education Multimedia and Hypermedia, 13(4), 391-404.
Fletcher, J. D., Tobias, S., Wisher, R. L. (2007) Learning Anytime, Anywhere: Advanced Distributed Learning and the Changing Face of Education. Educational Researcher, 36(2), 96-102.
Coming Soon: Training Evaluation Information on the ADL Website
ADL is planning to update its website (www.adlnet.gov) to include descriptions of the research conducted by the Training Evaluation Team, resources for evaluating training courses, and opportunities for collaboration. The research being conducted focuses on improving learning from Web-based training courses, understanding learning processes, and interpreting the meaning of training evaluation data. You will be able to view copies of recent presentations and request copies of publications. In addition, several opportunities for collaboration will be provided including a project examining the effect of course design on student satisfaction. Participating in this project will provide instructors with the feedback they need to improve student attitudes towards their courses. The site update will take place early in 2008.
Back to top |
