Friday 21 February 2014

Babbling of the Lomaproosa

I don't think anyone but Tässi will understand anything of this mind dump for I've still not managed to publish the old Lomaproosa anywhere. However, let this be published here, in a place where it is easily fetched.

I should continue the Lomaproosa - project. I have been doing thinking, and I am sure the stealing of Rajol's children would be... well, it would be a cliche, but a cliche that kickstarts the story. I'll edit it into something else if and when I see fit. The children, who Rajol doesn't know are his, are stolen to the lands of the south. Who are these thieves? And what the hell exists in the south?

Some time ago we decided with Tässi that although in the regular maps the north is in the up and west is in the left side of the map, in the maps of MERPG (available in the merpg.webs.com) the north is to the left and the west is thus to the down. This provides some sort of help for the cartographers (in other words, me, until Tässi is again available), but I fail to recall the exact nature of this help. Still, what is there in the south?

There should not be grand wizards, for they are supposed to be dead now that the Tuula was got rid of in the end of the previous text, and bringing them back would be as ridiculous as the Daleks, who are killed into extinction in the every other series of the Doctor Who. Pirates, which I think I spoke something of in the previous text, are also a ridiculous cliche, albeit a popular one. I should play the Assassin's Creed 4 through though until I write anything real of the pirates.

But what do I put into the south!? Who would be stupid enough to steal the children of the greatest hero the universe has ever seen. And how should the said hero understand the kids are his? And how should I get back to the guild he thinks views him a failure? And, of course, WHAT DOES EXIST IN THE SOUTH?

Let's try associations. In the lion king, there are the lions, who can be associated to the guild in my texts. Then there are the animals that aren't lions, but aren't the hyenas either. These can be associated to the common folks. Then, the hyenas, the awful guys. Who can they be associated to?

Some sort of nomadic tribe, I'd say. They don't know anything of the magic, which I'd say didn't work too well in the old text, but with their swords, horses and siege weapons they are extremely fatal to both villages such as the Vo Wacune (a tribute to the author who introduced me to the genre of fantasy, may he rest in peace) where the Rajol resides, and castles such as the guild's, resided by the assassins and their leader, Rajol's beloved Amrah. If these guys raze the Wacune (again with the tributes...), harass the guild into its knees and destroy half of the world that survived the mass destruction of the first story, I'd have a base to build the story on.

But what does motivate this tribe to attack the northern world? I could follow the Conn Iggulden's idea of the civilized world discriminating the nomads, but what for would the Amrah, the undoubted leader of the post-war world and one trying to unite the world, do bad things against them? No. Are they lacking in food, and also attracted by shiny metals the cities of the north are filled with? Yes, that could work. Food and gold, another cliche, but I'm not afraid of them, as you could see had I published the old text anywhere.

Thursday 20 February 2014

Documents and stuff

Again a good text from the 750words experimentation. Remember that these are more like brain dumps than actually researched and thought texts:

Well, this day was almost gone until I remembered that hey, I still have the daily 750 words undone!

The school today was awesome for once. It seems I'm doing the IM-project I've babbled about everywhere for the project-classes, which means I have to really put my hours to a book. I resist the "use excel for bookkeeping of hours" - clause of our instructions though. .xsl(x) is a stupid format in Windows, and completely useless in the real world. Currently I do the bookkeeping as a series of Clojure maps, with keys like when, count of hours, and what did I do. I think the git could provide the when and what is done - values, but it is useless in keeping track of the hours used. I think there exist real apps for tracking the used time per project, but since the school is too lazy to propose anything better than Excel, I'm also too lazy to experiment with stuff that is not guaranteed to be better than Clojure.

If I am allowed rant more about the office formats, I want to express my utter dismay about the state of the document-producing software of ours. First: MS Word is... well, if the document is expected to follow formal standard, then it is a nice tool, although producing software with strict, formal standard is dumb. For producing somewhat informal text that doesn't look like it is done by agitated baboon it is horrible. For that I use Emacs. Today I translated and edited the first IM-blogpost into a "starting report of the project" by writing finnish into an org file in my blogging folder, and then let the Emacs export it into something that doesn't yield "WTF is this format?" - responses. I hoped the odt would be enough, since at least in the Emacs of my mac the creation of pdfs is somewhat challenged, and producing docx:s from anything usable is not a walk in a park. Of course the response was "can't you... export it to pdf or something from that linux of yours?"!

Yeah, hold on a second, I'll just brick this macbook with a fresh install of fedora or something...

I spent ten minutes trying to get the latex->pdf - exportation working, and after failing that, opened the exported odt in the Libreoffice Writer, the only thing worse in producing text documents than Word, and exported the document into pdf.

I think the project document was accepted, and now I'm rewarded with credits for working on this thing! I have yet to check the official site where these projects are hosted, but at least I received no complaints as a reply to my email where I attached the report. I was also told the school could give me a hypothetical server in case the Heroku fails, but I am not really looking forward to that, since as far as I understand, they do not have too many public IP-addresses, and I am not absolutely certain I'm able to keep the security holes out of my code.

After giving my report I resumed hacking the server of this IM (which by the way needs a real name. I think the old MEsE is a stupid one, but this faceless "the IM-project" is also such.), and found out that the server doesn't correctly update the moment of the last call any time the client calls. This means that after five minutes from the logging in the client is officially timeouted with no way to reset the timeout. Let me tell you, the state will be the end of this world.

I should do designs of the client at some point. I've spoken of this project only as a better Skype without the VoIP, but that doesn't seem to be enough to convince some people about how awesome this project is, and neither does the fact that only thing I've got to show of it are some four blog posts and a screen full of Clojure. I have seen a few people that wanted to take part in this project if I switched the implementation language to something "sane", like the Java or C# we're being taught in the school, and in a general case I could do this (although assuring them the project will probably be doomed after it), but with this project I've actually tried them. The Java was lousy in the server side, and although the C# was somewhat verbose in the client side (compare XAML to s-exprs... :P), I could see me doing an alternative implementation of the client in C# sometime.

Monday 17 February 2014

Authentication scheme

Hello, friends! I am back in Kotka!

The Friend - authentication library of Clojure seems interesting, but because I'm a bad coder, who probably seems to hate sunshine, green grass and all other good things on this earth, I can't be arsed to learn to use this ready but large library that is somewhat complicated for my simple needs. I will probably earn a The Daily WTF - headline with this text, but a coder who has not done that, isn't a real coder :)

I am also too lazy to teach myself how authentication against Twitter's and Facebook's user databases would work. In the spirit of the Worse is Better, I will hard code Feuer (myself) and a few test accounts to the server and I abstract the user-authenticates? - function somehow nicely and I'll write the user management on either CouchDB or the previously mentioned SOME-services. When the authentication works, I'll write the web-API (REST-API?) (and test it somehow...) and begin writing the proof of concept of the client program.

It is unnecessary to dream of the clients and other stuff until the authentication works. How then should it work? In the beginning, the server is minding its own business, and then it finds a surprising login-request from the Compojure's level. The Compojure receives username and a naked password as parameters of the request. The password can not be hashed in the client, since if I remember correctly for the reasons I can't say I understood, the output of the hash-function is dependent on the environment it was called in. Both Java and PHP have given me different hashes in linux and windows for the same input in the past.

The password is immediately hashed with the sha512, and the userdatabase is queried with the username. If the username is found and the password matches, the user is authenticated and the client is responded with HTTP statuscode 200 and the sessionid, which is used for the authentication here onwards. Authenticationsystem updates the sessionid to the sessiondatabase with the current System/currentTimeMillis - value. Sessionid is also associated with the IP it was sent to, so that cracker guessing these ids can't get access to the session although finding an existing session-id unless they happen to be under the same public IP as the original owner of the session.

The rest of the requests do need the session id as a parameter. If the sessionid is found lacking, non-existant, the timeout has happened (the time from the last System/currentTimeMillis is more than 5 minutes (or 30000 millisecons)) or the requesting ip has changed, the request is dropped with response 403.  With every succesful request the moment of the latest request is updated to the currentTimeMillis.

What else? I don't know. The server's authentication scheme is hereby documented. Please tell me if you see something wrong with this.

Oh, and did I mention? I'm back in Kotka after a weekend in Espoo. This week has a possibility of being an awesome one. After my morning exercise (in other words, about three kilometers with a bike) and a couple of cups of tea, I'm ready to bleed some Clojure into the Emacs. Unfortunately I have the thing known as the school blocking my creativity here. Well, today I have a physics class, so the schoolday is in fact a bit more interesting than usually. And it isn't a long day in school. Unfortunately tomorrow I have to rise early for school, so I am not able to code tonight until 2 am. Hopefully tomorrow is different :)

Let me fill the word quota by expressing how dissatisfied I am with the gray the sky has been filled with for almost a month now. I'm sure I spotted the sun a couple of times in the January and the December, but this month has been a british one: rain, rain and not rain but still awfully gray. Luckily when I have been cycling the rain has kept away. In June a little rain isn't that bad (especially when I'm on a bike), but other times only weather condition worse than rain are the snowstorms.

I'm horrified how bad I'm in holding a train of thought even for as small texts as 750words requires. Then again, my daily thoughts don't take so much written space, unless I'm pondering on something locally revolutionary, such as the IM-project, which has filled almost three 750words texts now. Unfortunately all three of them have exploded into this kind of rubbish by the end.

Thursday 13 February 2014

~600 words on IMs and school

I have been doing 750words every morning this week, and now I have a text I feel could be published also here. It's not long, most of the rubbish I wrote in the original 750-entry has been edited away, but it has some interesting material concerning the yesterday's post. And I hope I'll publish more of them. The long text once in two months - method I've been doing here since discovering Yegge's blog hasn't been merciful on this blog, so I'll try to ape Swizec's way of writing shorter stuff at least weekly, and I'll try to wander outside the programming niche.

Last night I unveiled some new thoughts on this instant messenger - project I've been doing for a few years. I think the SMS is the best instant messaging technology we currently have, and I don't mean this as a compliment for those Nokia engineers who made this techonology two decades ago. I mean that as a compain to us fools who have had the microcomputers for three decades now, and the best thing for communicating with them is the IRC. Five years ago we also had the Live Messenger, which was barely better than writing SMSs alongside the coding, but the Microsoft killed the thing: first they created the Live Messenger 2011, which did have probably one improvement, but it also had dozens of stupid ideas, and the stupidest of them: after a year the thing was killed by Skype. Skype is somewhat usable as an AV-phone, but it is horrible with textual messaging.

From the mild attention I got (one, probably bot, more follower in twitter and one comment in the facebook) I can deduce this thing could gain some traction in the market. Unfortunately, if it gets, I really have to familiarize myself with the ad-networks, because I don't expect the free heroku-server to support more than five concurrent users. In the weird case I get the system to pay itself with the ads, I'll become one very happy panda (to utilize an odd metaphor). If the thing will generate revenue, I might also write clients for the three mobile platforms. There the ad-systems should be easier to implement than in the computer land, as would be getting the people to pay for the clients.

This project might also be applicable in the school. We have these weird classes known as the project classes, where us students are expected to come up with an idea (since we're too inexperienced to implement existing ideas for the local software companies, or something) and implement it. I think most of us will do some game or another, but since I'm not currently in the mood for developing such things, I'll try to get the yesterday's blog post accepted as a design for my first project. The only problem is: the projects are supposed to be done in groups with two or more people, and I'm actually a little afraid of opening my mouth on stuff that actually matters. So trying to sell this idea to one or two other people is hard, even if I was implementing my IM on more traditional tech. I imagine Clojure looks like ancient greece to guys who barely have their feet wet with C#.

I just asked in the facebook if anyone actually would be interested in this project. Let's hope someone responds out of serious interest, not out of "Feuer is ..." for some stupid reason, if I may add, "... deemed to be a wizard in programming. Let's hope the wizardness overflows to me also!" - feeling. For some reason the two (or three? or four?) people who I am certain would have any valid input (in either designing and programming sense) for this project don't even live in the Kymenlaakso.

Wednesday 12 February 2014

Ramblings of an IM-project

In case you haven't notice, Heroku seems like an awesome platform. They seem to offer Clojure & CouchDB for free, which means at least I can retry the old dream of mine: an instant messenger which doesn't suck. Last time I tried implementing that I had some troubles with the chosen platforms. Also, now that we've seen a few years without the prehistoric Live Messenger 2009, it might be a good time time to rewrite the specs for this dream-IM of mine. So, let's begin:

What must the IM be capable of

  • Sending messages from a peer to another in an acceptable time
  • Offline-spams have to be sent and received
  • Possible P2Ping files - but if I'll implement this, it won't be implemented performance in mind, and will be meant for sharing screenshots. Dropbox, and any other thing currently capable of doing this is lousy at it.
  • Message's fonts, colours, modifiers such as italics&bold, and the size of font used in the client must all be possible to change
  • Real group conversations won't be implemented at least right away. Possibly you could add multiple recipients, or make the client abstract the server's inability to do conversations away, but that doesn't concern us yet. I'm currently designing only the server.
  • The client should be scriptable - but only by the local user.
  • Inline styling should be possible with HTML
  • User's have changeable nicknames, personal messages, personal images (which have to be uploaded to the internet) and states. Quoting the last specs from the summer of '12, the states are
    1. ONLINE
    2. BUSY
    3. AWAY
    4. RETURNING SOON
    5. LUNCH
    6. FAKE OFFLINE
    7. REAL OFFLINE (ie. lack of any client polling)

How are the users handled? Does everyone just log into a server, and see everyone there, or is there some sort of friend-system? I'd say at first I add people to the server, allow them thus to authenticate, and then people add other people as friends, and only when they are accepted, are conversations possible. But I'll write the core messaging first and only when it works, I'll build this kind of stuff.

How do people authenticate? Damn if I know. If I don't find any nice authentication-system built (note to myself: this looks like a nice one) on clojure+couchdb, I'll probably replicate the Pröng's authentication system.

I'd love to use Pröng's user-table on this, but because I can't access the MySQL-db Pröng uses outside the arkku.net - server, sharing that table with an app living in Heroku would be a hard one to do securely. Besides, if I used Pröng's user-table, and this thing gained any traction, it'd mean I had to open up the Pröng also for anyone not being an user.

The IM should also support signing in from multiple places, and every one of them should receive all the messages sent to that user when asking. All the messages sent to the user after their last signing off from the client must be received when they sign on from any client. They could write some sort of filter that guarantees they won't receive the stupidest offline-messages (for example, when one boots up a computer last used a year ago, they should be able to opt-out of receiving the messages whose age is more than a month).

How would these filters be set up then? Since this is my dream-IM, I'm writing it of course in Clojure, which means the client has access to (read-string) and his lovely companion (eval). That means the user can customise their client as much they want (and even open the NREPL-connection to it, for serious modifications), and writing messages would be an emacs-like experience. Want to copy a text file from the interweb to a message? Write '(slurp "http://prong.arkku.net/")', press C-x C-e (customisable key-binding) and (if the replace is enabled) the function call is removed from the message, evalled with given params, and whatever it returns will be set to the message. Want to do maths? Write (* 33 (/ 44 (Math/sqrt 484))), press C-x C-e and that S-expr is replaced with the number "66.0". If the replace is disabled, the thing works the same, but leaves the s-expr as they were. Maybe non-replacing eval could be bound to different keys?

Message passing

The messages are simple data structures. They contain the sender's id, message, time-of-sending (for timezone-related things this will be the time of the server when it receives the message from the client, and this will by default be shown in the receiving client), receiver's id... and in the common case nothing else. User's font-settings can be queried from the server with the sender's id, Whenever user changes their font-prefs, the server will be notified and the new settings will be used only with the new messages. They don't apply retrospectively.

The machinery to route these messages from the sender to the client will be an interesting one. I think it would be best if every user had their own outbox, where the server-thread leaves the messages after receiving them with HTTP's PUT. In the server would exist threads blocking on read on every outbox, and once an outbox gets a message, thread reading it checks its intended recipient. If the recipient('s inbox) exists, the message would be popped from the outbox and pushed into the inbox. It it didn't exist, the sender would receive a "Recipient not found" - message into their own inbox, with SYSTEM marked as the sender.

I think the requirement for multiple clients online makes the client-polling the only real way of getting stuff from the inbox into the client. Clients poll every... fifteenth(?) second for the state of the inbox, and the server responds with a vector of messages (+ maybe the current font prefs for every sender, so they don't require their own queries) that the client can process with however deems fit. The default is to sort them by the timestamp, ask all the font preferences from the server, and print them to the conversation windows with the preferences applied. When the client polls, server finds all the messages that are not marked to be already sent for that particular client, sends them and marks them sent for that client.

In the beginning the in- and outboxes reside only in two refs in the memory. When the system is ready for production, I'll have added a system that sends immutable copies of both of these into the couchdb every once in a while, so that not too much of the messages will get dropped in case something bad happens to the server.

When should we expect results?

I'll probably set up a google code repo (or a github-repo, for Heroku requiring git I have no good reason to avoid the Github) when I have anything to show off, and paste the url at least to twitter. I'll paste it to here also if & when I'll write more about the project.

Saturday 8 February 2014

The Geometrician

Guess what - I made a game at long last! Actually, this isn't really a game, since at no point it shouts "Congratulations! You won!", the score just keeps going upwards until it overflows Java's long. It's more like a tech demo - which could be interesting to port into a few different mobile platforms.

But before I babble more about it, let me give you the link to the actual description and download site:

DOWNLOAD HERE

Requirements

The only real requirement is the Java7. It probably runs just by double clicking the jar in your favourite file explorer. If not, and java happens to be inside the $path, there are the scripts for both the windows and systems with usable shells.

Still, I won't guarantee this will work in Windows. JRE's WTFyness isn't that bad in unixes, and Windows's WTFyness isn't that bad with stuff that compiles to exes, but combined WTFyness of those two is so huge I can't guarantee anything. Especially if you have installed some jar-snatching program that'll wish to install this game to your 5 year old nokia phone.

Technical stuff

As everything I implement on my own time these days seems to be, this game is implemented in Clojure. It borrows some drawing and game-state-managing routines from the MERPG-codebase, and in fact began only because developing MERPG began to feel like work. I also got some inspiration from a certain game everyone around me seems to be attached to, and got interested in trying to guess how the state-structures were handled behind that game. As I said, this is really more like a tech demo than a game: the real work I spent honing the data structures (well, at at least a data structure…) and the functions that operate on them. The GUI-gamy-thing was an afterthought I hacked on top of the MERPG-code.

As you might expect, the world is represented as a two-dimensional vector whose width is screen-width/50 and whose height most of the time is screen-height/50. Every cells value is from the set #{:x :square :circle :triangle}, and initially it tries to avoid putting similiar cells next to each other, and usually fails miserably (:P). If the user selects two cells, the game checks in the first phase of the state-update-loop if the swapping of these cells is legal by first creating a coordinate-diffs matrice like this: [(- x1 x2) (- x2 x1) (- y1 y2) (- y2 y1)]. x's and y's represent the coordinates of the selected two cells. Then it makes sure you won't try to swap cells that aren't next to each other by checking that none of the values of coord-diffs are other than 1, 0 or -1. It also checks that you don't try to swap lean cells by checking the coord-diff isn't any of the following: [[-1 1 1 -1] [1 -1 1 -1] [1 -1 -1 1] [-1 1 -1 1]]. I acquired these magic matrices just by swapping cells in all four lean directions, and printing the resulting matrices in the Emacs.

After these base legality checks the game has only one more condition: the swap has to create a horizontal or vertical row of three or more cells of the same type. It is easily done: lets swap these cells and check if this condition is fulfilled. Though how this happens is interesting too: lets open the contains-horizontal-starys? - function (whose name is ridiculous, and has probably been born of typo from contains-horizontal-straights?, which too was a "it's midnight and the parts that do english in my brain have been dead for hours now" - kind of a name). The first if-not - clause in the code should probably be moved into the pre-conditions. After it the code checks if we were sent a whole world (i.e. a vector of vectors of keywords) or just a column of it (i.e. a vector of keywords).

If the former, we map our function against every column of the world AND every column of the transposed world, thus checking the both vertical and horizontal (i.e. vertical after the transpose) rows. If the latter, we partition the column with a function that just returns it's parameter, thus changing the [:x :x :triangle :circle :x :x :x :x] into ((:x :x) (:triangle) (:circle) (:x :x :x)). Then its trivial to filter seqs whose count is smaller than 3 away, and check if we have anything left.

Of course none of this applies unless there's two cells selected. If none or only one, click-handling routine returns the world unchanged.

After we've dealt with the selected cells, it's time to remove any straights of threes or more. Running the world through indexes-of-straights and flatten-index-results returns the indexes of all the cells who participate in these straights in an somewhat useable form of [[[x] [y y y y]] [[x] [y y]] [[x x x] [y]]...]. Remove-nice-rows processes through this index-seq, changing every [[x][y y y]] and [[x x x x][y]] to [[x x x] [y y y]] and [[x x x x][y y y y]]. In each pair of x and y it replaces the old value with :remove-me!, so that it doesn't have to adjust the rest of the ids to understand disappearing cells. When done with the ids, the remove-nice-rows counts how many :remove-me!s the world contains, and increments the score-out - atom ten times with that. After this, the function maps to the world a filtering, which removes the :remove-me!s from it.

Let's return to the core.clj's handle-nice-rows. When we have removed-nice-rows, its time to fill-reduced-columns. That function can't guess the required height just by finding the max of (map count the-world) (although, now that I think about it, why couldn't it?), so we need to tell the height by dividing the windows height with the cell's height. By the way: don't use the magic constants like the 50 is used throughout the code. fill-reduced-columns is a simple function: it just conses random cells on top of every column whose height<the-world's height.

After doing a round of removing nice rows and conssing random cells, we recur and do those as many times as needed. I added there a hard limit of ten recursions because I had infinite recursion in that function killing my REPL. Oh, and when there's no more to remove, we return to the :update - function, and return the new world inside the state map to the merpg.2D.make-game/make-game - thingie. We also return the score updated by the remove-nice-rows. This map is then sent to the :pre-drawqueue, which draws the selection-rects based on the lexical state (I think? Yell at me, if this isn't lexical state, please) and the score based on the state-map sent from the update-function. :post-drawqueue receives also the state-map, and calls to the Java2D-primitives the draw all the required things.

How do I handle the clicks? With bubblegum, of course. Inside the make-game I originally attached mouse-released - listener to the frame. Don't do that, for the origin will be at the top-left point of the window, se the first 10-or-so horizontal pixels will be used for window decorations. Then I moved it to the viewport-canvas, thus moving the origin to its rightful place, the most top-left point of the viewport, under the decorations. Inside this listener I get the location of the click, divide its components with the damned magic constant 50, cast them to int, and sent the to the make-game's parameter function mouse-clicked inside a vector. There I either deselect both selected cells, select the first cell, or select the second cell. The handling of the y-coordinate seems to be screwed inside the whole core.clj, which is the result of me needing to make space for the Score: %d - text and moving the whole world 50px downwards.

The future?

I think I've finished with the tech-babbling. I don't think there will be more real development by me on this project. I've had some (more than I actually expected, although the first one is a bug report from yours truly) input of this game: don't try to select cells outside the world (in other words: don't click on the Score - text), and instead of just drawing the current state of the world, the game should provide some sort of transition animations from one state to another. The resposiveness is also a problem: don't try to resize the window. I think it should be easier to just disable the resizing of the frame, and build a GUI for adjusting the frame's dimensions (or encourage people to change the values from the ~/.geometrician. It's not that hard.) than try to resize the world according to the resize events without screwing the state of the world up. If you wish to develop this thing forward, it's available in the Google Code for a reason.