Custom sorting for conlangs

Sorting a list of words into alphabetical order is usually a trivially easy task. But if your words are in a conlang, alphabetical order might be different from usual. Here's a Python class that, when instantiated with a list of strings (alphabet), creates a callable object that can be used as the cmp argument of Python's list.sort() method. alphabet can contain digraphs, in which case matching is greedy, and the CustomSorter will ignore any characters not found in alphabet, which is useful for separating pairs of characters that might otherwise resemble digraphs. If you're using Python 3, you'll have to wrap the CustomSorter in comp_to_key. (Hope preformatted text works)

class CustomSorter(object):
    def __init__(self,alphabet):
        self.alphabet=alphabet

    def __call__(self,word1,word2):
        comp=0
        if word1=='' and word2=='':
            comp=0
        elif word1=='':
            comp=-1
        elif word2=='':
            comp=1
        else:
            head1,tail1=self.separate(word1)
            head2,tail2=self.separate(word2)
            if head1==head2:
                comp=self(tail1,tail2)
            else:
                comp=self.alphabet.index(head1)-self.alphabet.index(head2)
        return comp

    def separate(self,word):
        candidates=self.Candidates(word)
        while candidates==[]:
            word=word[1:]
            candidates=self.candidates(word)
        candidates.sort(key=len)
        head=candidates.pop()
        tail=word[len(head):]
        return head,tail

        def Candidates(self,word):
            return [letter for letter in self.alphabet if word.startswith(letter)]

Some Original Reasons for Invading Earth

In some SF shows, it seems that Earth can scarcely get through a week without being invaded by aliens. The trouble is, the aliens are running out of reasons to invade. All the usual ones, enslaving the population, plundering the planet's mineral wealth, fleeing your doomed homeworld, converting the planet into a weapon of cosmic destruction, they've all been done. I'd like to suggest some alternatives for interplanetary despots in search of new reasons to invade.

Bragging rights

The guys back at home will be so jealous when they hear that we conquered Earth.

Turning the planet into a tourist resort

Beautiful scenery, fascinating cultures: Buy a SpaceTimeshare ® on Earth today!

Conceptual Art

This Happening challenges us to explore such themes as: What does it mean to be an alien? and Why do we really invade planets?

Dragées Those little silver coated sugar balls

used for decorating cakes. According to The Doctor, you can only get them on Earth. Have you any idea the price they fetch in Andromeda?

Parts of Speech in the Voynich Manuscript

As previously mentioned, I've been trying to do some analysis on The Voynich Manuscript. There are three basic schools of thought about what the language in the manuscript might be-

1. A cipher for some natural language
2. A conlang
3. Gibberish

As a conlanger myself, I tend towards the second hypothesis. If that's correct, the key to understanding it will lie in working out what the grammar of that language is. As a first stage, I've been trying to cluster words according to the environments in which they occur. The idea is that words that occur in similar environments are likely to have similar grammatical roles. Suppose we have a sequence of words A X B, then the tuple (A,B) can be considered the environment of X. However, since there are a lot of different possible environments, that would produce a very low signal to noise ratio. Therefore, for each environment, I found the set of words that could occur in that environment, and compared those sets between pairs of environments using the Tanimoto metric. I clustered the environments using a nearest neighbour approach (each environment belonged to the same cluster as its nearest neighbour). For each distinct word in the text, I then calculated the probability of it occurring in any environment found in each of the clusters described above. Comparing vectors of these probabilities with a Pearson metric, I performed nearest neighbour clustering. This initially produced 92 clusters, so I created a sequence of cluster memberships for each word in the text, performed clustering on that, and then merged clusters accordingly. I repeated this until no further reduction in the number of clusters was possible. I ended up with five clusters, about the right number to represent parts of speech. My results can be seen here (I do need to fix the formatting, though). A quick statistical summary is

Cluster	Number of words	Number of instances
0	4291	15790
1	1470	7040
2	747	3460
3	379	2270
4	4420	12142

But what to these categories mean? How do they relate to each other? That will be the subject of my next experiment.

The Great Debate

The hall was full of men, each of whom had one eye closed. The first speaker expounded at length that those who opened the right eye saw more clearly, and thus achieved a surer increase in wisdom than those who closed it. Another man replied that there had been a great many wise men throughout history who had opened their left eyes.
Another speaker said that a great many crimes had been committed by those who closed their left eyes. The reply came that some of the most notorious of criminals has been men that closed their right eyes. The first speaker countered that the perversity of such men was due to the fact that, having been trained in their youth to close their left eyes, by the time they closed their right eyes, they drew no benefit.
Another speaker said that those who closed their right eyes were constantly looking backwards over their shoulders, while those opened them looked ahead. His opponent said that those who closed their left eyes were so obsessed with going forwards, they never looked around to see if it was the right way to go.
A man rose to his feet to say that it was necessary to close the right eye, so that the vision of the left should not become clouded with illusions. At this, another jumped up and shouted angrily that to say that the things seen with the right eye were illusions was a wicked lie, and it was more than likely that all the things he claimed to have seen with his left eye were wicked lies too. A younger man then shouted out that the right eye saw nothing at all, and that all who claimed to see through it were merely repeating what others had told them.
Thus the debate continued, becomming more and more heated. But I left the chamber with both eyes open.

Downloading "A Game of Thrones" - A British Perspective

I tried to watch Game of Thrones and this is what happened - The Oatmeal

This webcomic has been causing a fair bit of controversy on the Internet recently. Now, I'm all in favour of creators getting paid, and I know somebody who's involved in the production, and certainly deserves his cut, but here in the UK there's another problem. "A Game of Thrones" is broadcast by Sky over here. Sky's owners also own some tabloid newspapers, which are at the centre of a major scandal at the moment. Their journalists have been involved in phone hacking and corrupt payments to police officers on a massive scale. So it looks like, over here, even if you watch the programme legally, you'll still be giving money to a bunch of crooks.

Captain Zeno Goes Faster Than Light!

We humans don't like the word "impossible", which is why, no matter how unlikely it is that the neutrino anomaly will lead to faster-than-light spaceships, I, for one, have my fingers crossed. Let us imagine, for the sake of an SF thought experiment, that Captain Zeno is on the bridge, ready to begin humanity's first voyage to Delta Pavonis. He's plotted his course as accurately as possible, but over 20 light years, the margin of error is pretty vast. Worse than that, when you're going faster than light, you can't see where you're going. How can he steer?

Fortunately, there's an answer. Halfway there, he stops, checks his position and corrects his course. After half the remaining distance, he does it again, and keeps doing it until he's close enough to Delta Parvonis to go the rest of the way at a normal speed.

I call this Zenonian Navigation, after Zeno of Alexandria, who claimed that motion is impossible, because before you can go anywhere, you first have to get halfwat there, and before you can get halfway there... Probably the most obviously wrong idea in the history of maths.

The Voynich Manuscript

As you may have noticed, I haven't been posting for a while. I changed jobs, and while I'm enjoying my new job, I'm working longer hours so I haven't had as much time for things like blogging. I also did Peter Norvig and Sebastian Thrun's online AI course and as you can see here, I did pretty well. Doing the course gave me some ideas. For a while, I've been interested in the Voynich Manuscript, an enigmatic document dating from some time in the 15th century. It's written in an otherwise unknown script, in a language that nobody has been able to identify. Judging by the illustrations, it seems to cover such subjects as alchemy, astrology and herbalism. I personally suspect that it may be written in a conlang, which, if correct, would make it one of the earliest known. I think that by using Artificial Intelligence techniques I might be able to have a go at cracking it. In particular, I'm planning to use a combination of clustering and Hidden Markov Models to try to work out the syntax of the language. In the process I'm working on a Python library for Hidden Markov Models, which I hope other people might find useful.