3 posts tagged “memory”

If the light of a thousand suns
were to rise in the sky at once
it would be like the light
of that great spirit.
— भगवद् गीता

I remember when we were so young,
you embraced my fears and made me strong;
but never did you actually hold my hand,
your silence no one would understand.
— Dream Theater, Speak to Me

I discovered this weekend that Fravia+ passed away about two months ago; I had read about his illness some time ago, but somehow missed the announcement of his death until now.

For those of you unfamiliar with this character, Fravia was one of the old school reversers (as in reverse engineering); or "hacker", in the sense that I personally identify with. In those days, knowledge was handed down from the great wizards like +ORC and others, but it was Fravia that truly brought the message to the masses (you may find his old site archived in various places). This was not merely about "cracking" software copy protection; but about taking things apart, understanding the way they work — and not just software, either. Later, he turned his focus to the meta-art of searching; seeking out information, wherever it may be hidden. His eccentric tone, which some no doubt found to be pompous and patronising, was something that endeared me to him from the very beginning; and the communities that sprung up around the gardens he planted were true jewels gleaming in the darkness of cyberspace.

Unfortunately, as time moved on, I lost touch with these communities, as I have lost touch with so many others. I still feel a strong connection to them, but I ceased participating in the interactions for whatever reasons, as my focus moved on to other places; and this is really what this post is about. Somehow, despite my efforts to the contrary, I'm unable to cling to everything I hold dear; like trying to gather the sea into one's hands, it just flows through my fingertips, and I don't know what to do about it. In some ways, this is similar to another problem I have previously described; dealing with ideas that are too big to hold in mind all at once. Yet, this is not quite the same issue; this is more of a social issue, wrapped up with issues of time and concentration. I somehow need to become my future self, without losing that which comprises my past self; to gain new understanding and insight, without losing that which was previously important.

And so, I find myself conflicted and disquieted; and most of all, mourning the passing of a great man, someone I would have liked to number amongst my friends, even if it were not so. Sail well, dearest Fravia, you are someone who will not be forgotten quickly or easily; and to those who perchance remember an old stranger, fellow traveller, or friend, I miss you all somehow, somewhere...

I've recently been grappling once again with an old problem: how to manage the development evolution of an idea that's too big for my mind to consider all at once. Technology has provided tools to help deal with this problem in general, mostly in the form of enhanced communication channels and "external memory" (ie. storage), as well as information processing tools to sift through external memory. Unfortunately, these tools only go so far; for one thing, in order to write down information or otherwise store it externally in some form, you need to be able to encode the information (say, in English). This works great in many cases, but is of little help in a situation where the ideas and concepts are not sufficiently crystallised in order to be able to encode and communicate them. For ideas with sufficiently small scope, it's not a problem to keep the idea in my mind over a period of time, as I slowly refine and crystallise the idea to the point where it can be communicated to others; but every now and then, something comes along that's so huge that I can't keep it in mind all at once.

One solution that some people resort to is encoding partial fragments of the idea independently. Unfortunately, this leads to a breakdown in cohesion and coherency; instead of a single coherent idea, you now have a sprawling mass of interrelated ideas that don't fit together so well, which really isn't a good substitute for the real thing.

Unfortunately, I don't yet have an answer to this; the particular idea that I'm working on (which will probably turn into a blog) is simply too important to break down into separate ideas, as it just won't have the necessary impact in that form. Every time I pick the idea up again, I realise that I've lost my grasp on various aspects of the idea, so it seems like I can't make any progress; as soon as I develop one aspect, I lose what I've developed on another aspect.

Anyhow, I guess I'm not really expecting a solution to any of this, but I thought I'd throw it out there while I'm banging my head against the wall.

At some point in my youth, I discovered one of the sad truths of history; mankind has discovered a great many things, and then lost that information again, repeating this tragedy over and over throughout the course of history. In some cases we can only speculate at what may have been lost (consider the construction of the pyramids, for example), in other cases we can confirm with relative certainty have rediscovered the lost information or technology, but nevertheless the historical periods in which mankind was without this information are depressingly long. Of course, to my naïve thinking, computers clearly provided the solution to this problem; store all of the information digitally, and never discard it; hard drives are cheap, Moore's law blah blah etc.

Unfortunately, that's only the beginning of the problem, not the end. For starters, we still don't have the storage facilities to store all of the information produced, although we are in a much better position to deal with it; storing books is no problem, for example, but storing all of the audio/visual information produced is a much more difficult task. However, even as our storage capabilities increase, another problem looms; it's no use storing information if you can't retrieve it again later, and retrieval by location is a severely limited mechanism. What we really need is retrieval by description; in other words, searching and "filtering". In the early days of the internet, even searching this "massive" distributed network was still a manageable problem; and so various search engines sprung up, indexing all of the content on the internet -- or at least, all of the content reachable via the hyperlink graph. As the Internet continued to grow, the difficulty of indexing the internet grew along with it, and today search engines like Google rely on truly staggering farms of storage and indexing servers to keep up with it all.

Here we meet the real problem; they *don't* keep up with it all, anymore. Despite the massive infrastructure being applied to the problem, Google (and the other search engines) still only manage to index a small fraction of the Internet today. They offset this by trying to make sure they index the "important" stuff, but the net result is that we're still losing information every day. Most pages will eventually fall off the internet if left undiscovered, but even if the information remains on the network, it's of no use to anybody if they can't actually retrieve it; and to retrieve it, you first need to find it.

There are parallels to this problem in other areas; for example, the information storage mechanisms of the human brain: our "memory". People often have the experience of struggling to retrieve a particular piece of information from their memory; it's still there, but they have to wait until something "jogs" their memory before they can finally retrieve it. Research into the functioning of the brain is only beginning to give us an inkling of how memory storage and retrieval actually works on a neurological level, but certainly the high-level process seems to have many of the same problems that our external information storage systems have. In passing, it's interesting to note some of the scientific theory associated with my intuitive feelings about some of these issues; without departing into complete mysticism, you may find it interesting to look at Holonomic brain theory, as well as the Holographic principle, and maybe even take a look at some of the crackpots trying to unify all of this. I'm intuitively expecting something scientifically sound to emerge in this area, but we'll have to wait and see how that turns out.

That digression aside, I'm not sure where this leaves us. The closest biological model we have seems to suffer from the same problems, so that doesn't help us; and I'm not sure where else we have to look to. Is there some research in this area that I've missed? Some potential new technology that might solve the problem? Let me know if I'm missing out on something.