Tuesday, June 21, 2011

The Glory of the Hunt

People care about their files. They are attached to the personal systems they put in place to find what they need, and become anxious if they are taken away. The file system is guarded territory, as fiercly protected as a parking stall.

It became clear years ago that it is not enough, as a records professional to build a pure filing system - elegant in design and intuitive to use. If I have not engaged the very human customers where they work and live, the system will fail.

Quoting google advanced help, "Search engines use a variety of techniques to imitate how people think and to approximate their behavior. As a result, most rules have exceptions. For example, the query [ for better or for worse ] will not be interpreted by Google as an OR query, but as a phrase that matches a (very popular) comic strip. Google will show calculator results for the query [ 34 * 87 ] rather than use the 'Fill in the blanks' operator. Both cases follow the obvious intent of the query." This means rather than being straightforward, search support services spend a lot of time getting inside the heads of searchers, to help them get the results they want (not what they say they want).


So how do we stow away and find things? It could be that our personal classification systems and style of hunting (foraging) are as established as our hunter-gatherer brains. Web and corporate Enterprise Content Management (ECM) systems have vastly extended our reach, but our searching instincts have not changed.

I will suggest that classifying and searching, tagging and recalling successful hunts, are part of our instinctive heritage. Steven Pinker in his book the Language Instinct, proposes fifteen instincts that are hard-wired in to all of us. Two of these instincts relate to searching and classifying:

4. Mental maps for large territories.


11. A mental Rolodex; a database of individuals, with blanks for kinship, status or rank, history of exchange of favors, and inherent skills and strengths, plus criteria that valuate each trait.

Imagining our forbear's steps, I imagined my ancestor following a familiar trail, noting edible plants along the way. She would retrace her steps later, when she knew the harvest would be ready; wild carrots in the summer, cattail tubers in the fall, and rose hips through the winter. She would have identified and classifed the edible plants, and remembered the trail to get there.
I couldn't find a comparable image on my google search, so I scanned my own, I did learn a little about the foraging habits of water pipits, larval green lacewings, and modern human urban foragers.   
So in many ways, classifying and searching is instinctive. We care about the results of the hunt, and not just for the practical purpose of getting the job done. This is personal.

You know what I am talking about. Anyone in our business will have hit a tough search that evades early detection. We dig in ever harder, searching out the obscure places where it might have been put. To place that record in our bosses hands, sweet.
The hunt is valued. A swift and successful hunt gives value to the organization. A hunter who provides consistent results is an asset; not just from an empirical, practical point of view, but at an instinctive, visceral level. I suggest again that if the GARP(C) principles were to be ranked, Availability is at the top. Not so say that the rest may be discarded. Together, they complete the framework for a robust records system.

When converting to a new file structure, be respectful of people's need to find their stuff. Anticipate the anxiety that accompanies change, and prepare for it. Make sure they have time to orient themselves to the new system, and reassure them that the materials they need daily will be at hand.

Monday, June 20, 2011

Advanced Search in the New Age

I've struggled with this subject all day. It's hard to pin down why. I enjoy running a great search, and I'm good at it. I think it must be because many of the tips and tools I'm highlighting, are as natural to use as breathing. It's tough not to step over my own feet when laboriously laying out all the steps. The problem these days - on google at least - is not the absence of results. The problem is too many results from a simple search.

When the internet was new, my girlfriend showed off her google search for "superman". Her son was a comic book buff, and she and her son marvelled at the speed of the return; four hits. When she demonstrated for me a couple months later, we found twenty sites. And goggled. My, how the internet was growing in leaps and bounds. Today, a google search on the same term gave me 168 million hits. My mind boggles at that number. In truth, I won't look past a couple pages. The likelihood that I would find a significant result any deeper is just too small.

To make sense of this mass of information at our fingertips has made search an art. Find a term significant and unique enough to bring back the result I need, but not so narrow that it filters out the gold. A way to develop this fine touch is to start with the narrowest search you can think of. Try enclosing your google phrase in quotation marks. If you get no results, broaden your search ever more slightly. After  a while, you will develop a fine touch. Here are two google searches I conducted recently, that required several google tries to find me what I wanted:

  • There is an archeaological dig on she shores of Galilee, profiled by the Naked Archaeologist . There's evidence of a fishing industry, and early Christian activity. What was the name of the dig? I'd forgotten. I searched filtering only Naked results, and found the name of the fishing village. I then broadened the search for Bethsaida. Google corrected my spelling, of course. And there it was, in satisfying detail, the results of a dig briefly profiled on Naked.
  • A student mentioned BlueCielo as an Electronic Content Management (ECM) tool that manages engineering drawings in Computer Aided Design, (CAD) format. After checking out the official site, I wondered what the community is saying. I used advanced search to limit the results to "Discussion:". Google found me what I wanted, but the discussions were empty. What is it with the community? Do they sit around the water-cooler to chat? Is there no twitter feed, no chatter, no casual trail for me to follow? I remind myself that this is not all bad. People talking. In person.
Before I go any further, I'll briefly discuss the differences in a corporate electronic file search and the world-wide web. Most of the time when conducting an internal search, you are looking for something you know exists. You either put it there yourself, or it is a manual/report/document that you have referred to in the past. You resort to search because you've forgotten in the webonious structure where you've last laid it. If it is an Explorer search, a panting dog may wag his way through to help you.
Failure to find the document you are looking for will likely lead to a few hours of frustration. Because unlike a google search, you must find the document of your recollection. The average information worker spends 8.8 hours a week searching for information. (Ref. The Importance of Enterprise Search, slide 13, IDC Hidden Costs of Information Work (2005) ). It is therefore critical that the electronic information management system that you select is capable of masterful (and swift) searches.

Similarly, in an e-discovery (may you never be blessed), search results must be consistent and complete. Correspondence has an annoying habit of referencing past correspondence. Does the search find both? Missing key documents will challenge the comprehensiveness of your records, and the reputation of your corporation.

Now that my little rabbit trail is done, I can go back to discussing advanced searching techniques. Most of these help you narrow your search. As I've mentioned before, a dearth of answers is not our problem. If you don't believe me, try running a search for "report" (5 billion hits on google). Advanced techniques include wildcard searches (named after the Joker in our decks), boolean searches (AND, OR and NOT), and a few more I found during my google search today; fuzzy, proximity and range. Though google calls these features by another name, you can practice wildcard and boolean in advanced search. Google has a great help page for advanced searchers.

Wildcard is replacing a character or range of characters with a symbol ("*" on most of the systems I looked at today). I would have found Bethsaida sooner if I had typed Bet*da. I'd mistakenly looked for it as Bethseda.

The boolean link I've referenced is a great tutorial that graphically illustrates the different sorts of results you get. Google uses these same boolean terms, so check out the results. AND and NOT gives you a narrower result. If you care to check out my internet presence, try the google result "jgnat -java" (jgnat NOT java).

I've begun reading up on fuzzy, proximity and range when reading the features of Apache Lucene, an open source search engine. I won't try and pretend to explain them fully here. Range can be very helpful to narrow to a period of time, (i.e. Business Plans for the first three quarters of 2009) and tricky to get right. Fuzzy claims to bring back words that sounds like (but are not spelled like) what you've asked for. This might also have helped me find Bethsaida.

It is very worthwhile as information professionals to master these techniques. Information workers need all the help they can get to find their information swiftly and consistently. Be the expert, and we will demonstrate our worth to the organization many times over.

Friday, June 17, 2011

The Glory of the Hunt - Searchable Records

Trending topics in the records world these days are e-discovery, security, and collaboration. I propose however, that the most valuable skills record-keepers have to offer in the 21st century is the the power of search.

Search falls in the Generally Accepted Recordkeeping Principles (GARP (C)) under "Availability", An organization shall maintain records in a manner that ensures timely, efficient, and accurate retrieval of information.

Why do I claim this is skill is valued over the others? Because any records system passes or fails on it's ability to deliver. If it can't promise to give your information back when you need it, why would you use it?

Over the next few days, I'll highlight the power of search; some of the advanced tools that we should be familiar with as records professionals, why the hunt is its own reward, and some technical marvels on the horizon.

Monday, May 23, 2011

Paper Reduction - It's the Green Thing to Do

I got asked one day how many sheets of paper in a tree? (Over 8,000). It's the new benchmark of green-y; how many trees you can save from the chopping block. Hold off the printer, retire the copier, and read it online. As I tour offices with my Information Manager's eyes, I don't see stacks of paper any more. The signs of disorder are hidden away on e-mail and shared drives.

Reducing paper production - especially at source - is a great way for a company to go green and save money besides. I've posted a paper savings estimator on my site (www.platosgnat.com) that will show you how many trees you've saved and how many greenhouse emissions you've reduced. I've made counting easy, too. Add up the number of boxes of paper your area purchased last month or last year, and chunk in the number.


Using Shel Busey's good-better-best ratings, let's compare ways a corporation can reduce paper consumption.

Good
Switch to 100% recycled paper. Reduce the number of copiers in the office, and eliminate personal printers. You might consider an all-in-one printer/copier/scanner. Be sure you have procedures to confirm that the scans are authentic (CAN/CGSB-72.34), and have a plan where the scans will go. No use scanning if you have no place to put it.

Better
To get paper off the floor and everyone working online, scan your high volume collections. This will get you your space savings (if staff don't find new ways to fill it), but it won't save you a tree. Because to scan, you have to start with paper somewhere.

Best
Work with your IT and forms design to allow external customers to send you information electronically. PDF forms can be designed to submit directly in to your database. Set up e-pay with vendors who submit high volume transactions for payment (i.e. phone bills). Not only does this save the tree from source, you will start to see real savings in entry and handling of the information.

Monday, April 4, 2011

The Fate of Africa - A History of Fifty Years of Independence

I've just finished the above titled brick-of-a-book by Martrin Meridith, who provides a dispassionate, and well-documented, account of Africa's recent history. I am driven to know more about Africa first of all because of my son-in-law's backgroound, and also to get a sense of our world-wide humanity. To understand Africa, it's heroes and faliures, is to understand ourselves. To turn a blind eye to failure is to guarantee repeat.




I'll first quote another reviewer, to get a sense of the breadth and depth of this book, "So many bad books receive extravagant praise that it's hard to do justice to a msterpiece when one finally appears. And the Fate of Africa is a masterpiece...This is living, breathing history that matters." - Ralph Peters, New York Post. In case after case, country after country, the author describes how the Big Men of Africa squandered the wealth and potential of their countries for personal gain. First-world countries bungled and amplified the problems by failing to grasp the depth of Africa's issues and rather imposing their own agendas and fears. As I descended through despair as so graphically outlined in this book, I wondered, when would hope arrive?

I live by hope. I must believe that goodness prevails. Yet over and over again I witness in this book persons of integrity, intelligent and brave - harassed and destroyed.  The bullies won. They succeeded in raping their countries to the brink of ruin, and recovery is out of sight. How can the common person in such an environment dare to hope for a better day, that they will enjoy the fruits of their labour?

There can be no other way than hope and reconciliation. Yet it will take tremendous effort and insight - not by individuals - but by a collective world - to turn things around. It means being persons of integrity, altruistic, ready to spurn personal gain and power for the greater good. It means laying down treasured ideologies, trusting all that is good, and banishing the crooked.

For the rest of my blog, I'm preserving the quotes from this book that moved me, lest I forget.

[On the reckless speed that independence was granted to the Gold Coast] "A new governor, Sir Charles Arden-larke, was despatched to the Gold Coast in 1949, with the warning that 'the country is on the edge of revolution' and with instructions to implement a new constitution...in order to avert it. The new system of government was regarded as being in the nature of 'an experiment', one that could be carefully controlled and monitored...The reality, however, was different. One senior British official involved in the Gold Coast experiment later described the process as 'like laying down a track in front of an oncoming express.'" [Revolution's explosive power came from years of repression by the colonizers, who treated its citizens as inferior. Independence was granted late and poorly, not allowing for a steady transition of power] (p. 13)

[The error of divine certainty] "The architect of grand apartheid, Hendrik Verwoerd, a Dutch-born ideological fanatic, cast himself in the role of a leader chosen by God. 'I do not have the nagging doubt of ever wondering whether perhaps I am wrong.'" (p. 178)

"...a Methodist bishop, Abel Muzorewa, warned of the deep undercurrents of bitterness rising among the African population, 'the repressed fear, restless silence, forced tolerance and hidden hatred'." [The ruling leader, Smith, respondid with further repression] (p. 321)

"In sum, what the [World] Bank advocated was political liberalisation. Economic success, it maintained, depended to a large degree on effictive and honest government, the rule of law, open economies and policital democracy....."People need freedom to realise individual and collective potential...I fear that many of Africa's political leaders have been more concerned about retaining power than about the long-term development interests of their people. The cost to milions of Africans...has been unforgivably high.[Barber Conable]" (p. 376-377)

[Pastoral letter, eight Catholic bishops of Malawi on Banda's dictatorship  in Malawi ] "...This is most regrettable. It creates an atmosphere of resentment among the citizens. It breeds a climate of mistrust and fear. This fear of harassment and mutual suspicion generates a society in which the talents of many lie unused and in which there is little room for initiative.[Banda responded by declaring the letter to be a seditious document. Riots and violence followed] (p. 407-408)

[Frederick Chiluba at his inauguration] "The Zambia we inherit is destitute - ravaged by the excesses, ineptitutde and straight corruption of a party and a people who have been in power for too long. When our first president stood up to address you twenty-seven years ago, he was addressing a country full of hope and glory. A country fresh with the power of youth , and a full and rich dowry. Now the coffers are empty. The people are poor. The misery is endless." (p. 411)

[Angola in 1992] "Overall there was widespread relief at the respite from war, but scepticism about wheter it would last. 'Will it be like 1975?' a market trader asked. 'It is not the people who make war, but the leaders.'" (p. 605)

[Mandela at the opening of the secont session of parliament, South Africa, 1995] "The government literally does not have the money to meet the demands that are being advanced. Mass action of any kind will not create resources the government does not have. All of us must rid ourselves of the wrong notion that the government has a big bag full of money. The government does not have such riches. We must rid ourselves of the culture of entitelement which leads to the expectation that the government must promptly deliver whatever it is we demand." (p. 651)

[The Truth and Reconciliation Commission (TRC) on the African National Congress (ANC's) claims of self-exoneration as a 'just war' on aparteid] "A gross violation is a gross violation, whoever commits it and for whatever reason. There is thus legal equavalence between all perpetrators. Their political affiliation is irrelevant." (p. 657)

[Archbishop Tutu on the dangers of a new tyranny in South Africa] "We can't assume that yesterday's oppressed will not become tomorrow's oppressors. We have seen it happen all over the world, and we shouldn't be surprised if it happens here." (p. 659)

[Mandela nearing retirement] "I am nearing my end", he told Afrikaner students. "I want to be able to sleep till eternity with a broad smile on my face, knowing that youth, opinion-makers and everybody is stretching the divide, trying to unite the nation. HIs legacy was a country which had experienced greater harmony than at any previous time in its history. (p. 664)

Thursday, March 17, 2011

Open Data and the Information Manager

The 1995 movie, "Lorenzo's Oil" tells the true story of a father who desperately searched for a cure for his son's (Lorenzo) terminal disease, Adrenoleucodystrophy (ALD). At the time, sufferers did not live past five years. Augusto Odone scoured the local medical library for answers and sponsored a medical conference to bring together experts in the field. It was at the conference that he first found a glimmer of hope. An oil - oleic acid - was able to destroy long string fatty acids, Lorenzo's Oil. Young Lorenzo lived in to his thirties, passing away in 2008. The oil is used to treat asymptomatic boys with ALD and has been shown to prevent the disease from taking hold.

For the sake of my topic, I'm focusing on the value of bringing together experts and information. I'll discuss the value of privacy controls another day. With the advent of the internet, we have even greater opportunities to share discoveries. Hence, the open science data movement.

The idea is to freely share without restrictions from copyright, patents or other mechanisms of control. The internet by design fosters sharing. Ever since it's explosion, however, controls have been developed to prevent unauthorized disclosure. When it comes to my personal banking information, I fully agree. But it could be argued that these restrictions do not fit well where we can benefit from free and open sharing of information.

"Numerous scientists have pointed out the irony that right at the historical moment when we have the technologies to permit worldwide availability and distributed process of scientific data, broadening collaboration and accelerating the pace and depth of discovery…..we are busy locking up that data and preventing the use of correspondingly advanced technologies on knowledge [1]

Here is open scientific data that I have sourced in th epast:

African Plant Database: (learning more about my beloved sprouting Gazania)
The database currently comprises 1,86,948 names of african plants with their nomenclatural statuts. Data capture, edition and broadcast are the product of a collaboration between the South African National Biodiversity Institute, the Conservatoire et Jardin botaniques de la Ville de Genève, Tela Botanica and the Missouri Botanical Garden.

Jstor
 ...trusted digital archive of over one thousand academic journals and other scholarly content. We use information technology and tools to increase productivity and facilitate new forms of scholarship...

In other realms, we have Facebook founder Mark Zuckerberg saying that privacy is "no longer a social norm" and a movement for open data of government information. I think it is no accident that is is the data community pushing for these reforms. These are the master-chefs of information, itching make new use of these rich sources. To be clear, open data is far more than publishing reports and final results. These new data-chefs want access to the raw data to manipulate and present in new forms.

Here in Canada, the open data movement has sponsored Change Camps and convinced the City of Vancouver to provide open data. It was at a Change Camp here in Edmonton that I was first made aware of the Open Data movement. The City of Edmonton provides raw transit information to google, for instance. Mobile Edmonton travellers can now plan routes through google. There are American examples how new open data applications are improving government. Microsoft has sponsored a competition to find new uses of Vancouver's data, like Vantrash to help users schedule and track their trash days.

An additional benefit of open data is to improve openness and transparency of government activities to the public. Where there is openness there is trust. For instance, I found an online air quality index for a neighbour concerned about a bad smell. I follow the blogs of Paul Levy, former CEO of a Boston Hospital and advocate for transparency of clinical outomes,. The Japanese government has taken some criticism this week for not being more forthcoming on developments at the failing nuclear plant. Open, honest and timely information reduces public anxiety.

Organizations are meeting he challenge of open data without infringing on rights. The international Open Data Commons has developed a set of licensing standards that an author can cite to protect their interests while providing free and open use of their data. The Commons has also provided definitions and principles to guide the open data movement:

"The Open Knowledge Definition (OKD) sets out principles to define ‘openness’ in knowledge – that’s any kind of content or data ‘from sonnets to statistics, genes to geodata’. The definition can be summed up in the statement that “A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the requirement to attribute and share-alike.”

Why bother about openness and licensing for data? After all they don’t matter in themselves: what we really care about are things like the progress of human knowledge or the freedom to understand and share.

However, open data is crucial to progress on these more fundamental items. It’s crucial because open data is so much easier to break-up and recombine, to use and reuse. We therefore want people to have incentives to make their data open and for open data to be easily usable and reusable — i.e. for open data to form a ‘commons’."

The Open Data Commons gives a special mention to government data:
"Open government data and content is material that is:
  • “Open” as defined by this site’s Open Definition– in essence material (data) is open if it can be freely used, reused and redistributed by anyone.
  • Produced or commissioned by government or government controlled entities."
In 2007, American advocates have developed eight principles of Open Government, and here in Canada, Open Data proponents are calling for the resolution of issues around licensing. In the realm familiar to Information Managers, the eighth principle of GARP is Transparency (A counterbalance to the third Principle of Protection).

In Alberta, the principle of open government is balanced against the citizen's right to privacy in the FOIP Act [2]. Most people are familiar with their privacy rights in this legislation, but the Act also allows open access to government information (sections 2 and 6)


"The purposes of this Act are...to allow any person a right of access to the records in the custody or under the control of a public body subject to limited and specific exceptions as set out in this Act... An applicant has a right of access to any record in the custody or under the control of a public body, including a record containing personal information about the applicant."

Our Information and Privacy Commissioner has spoken of openness and transparency during Right to Know week. At a seminar that same week, the City of Edmonton spoke of the benefits of routinely disclosing public information. Our snow removal crews took a big hit this winter, but I found their online schedule and status updates did a lot to relieve my concerns. I had a bet going with my husband if our "arterial" would be cleared by deadline, and it was.

So there you have it. For oppenness in information as mundane as trash days, to clinical outcomes in our health care systems, open data fosters collaboration, informs in new ways, and builds trust. As an information manager, keep an eye out for data sets of information that is already routinely disclosed in other formats. Introduce the principles of open data to your clients, and give examples how open data can improve transparency for their organization.

[1] John Wilbanks, Executive Director, Science Commons
[2] Freedom of Information and Protection of Privacy Act (Alberta, F-25)

Thursday, February 24, 2011

Tagging Touches Records Management

On February 16, our local chapter of ARMA hosted Chris Izquierdo of DevFacto to speak about folksonomy vs taxonomy . A good argument was made to hand the reins of classification over to the users (folksonomy) with perhaps some controls in the background (taxonomy). When it was all over, a friend turned to me and asked, "What is tagging?" Goodness, I realized. I've been playing around with the internet for so long, I've forgotten that these concepts are new to some. Tagging works great with electronic documents and is virtually impossible to replicate in the paper world. So let's spend a few moments finding out what tags are and how they can help classify and search for our electronic records.


Tag has many definitions, and most have to do with adding more information to
an object. The tag on my new headband gives me information about the manufacturer. In the past, when a user sent records to file, they may add a file number to the corner. The user was restricted to the classification plan. The file number is a sort of tag, but it is limited in scope.

On an internet form, however, a as many tags as needed can be attached to a document to help the user sort their entries, and readers find what they are looking for. This blog, for instance, offers tags in the field marked "Labels:" and shows up on the bottom of my post. The software remembers what tags I have used in the past, so as I start typing, it offers me choices based on past use. Over time the tagging becomes ever more consistent to the way I classify things.

I tag my posted recipes, too. The site offers sort and search on my tags, offering new ways to find and manage my recipes.

Tagging photos is a huge hit. Flicr was a pioneer of photo tagging in
 2004, but we can now tag on virtually all digital photo sites including Picasa and Facebook. Internet software is now sophisticated enough to recognized faces on a photo, and will often prompt the user to tag the person. I can now search for all photos that contain the tagged face of a friend or loved-one.

Besides personal tagging, there are tag cloud generators that automatically build graphical representations of your tags. The good generators should highlight by significance (i.e. tags with greatest weight are bigger). There are free tag generators available. I regularly generate a tag cloud of my blog. Every word is a link to the source that the reader can follow. http://www.tagcloud-generator.com/  Try it out for yourself.


So how does this change the Records Management world? Internet savvy users may demand the option to tag their documents for later retrieval. Tags (keywords) can be added as optional metadata entries in Electronic Records Management applications. In the future, tagging might replace formal taxonomies as the primary cataloguer of electronic records. You may offer new search options by offering tag clouds of frequently searched content.
 
The classic paper file manual, a true brick, able to stop a steel door in it's tracks, might soon be a relic. But people are still driven to classify and organize. We have the opportunity, as RIM professionals, to offer new ways to look and search; to guide our users on how to tag records for easy retrieval, and to show off our information in new ways. In the future it won't be just the linear thinkers who rule the day; our visuals and creatives have the opportunity to find what they need, their way, too.