Expanded Thought Map

Here is an expanded thought map, as you can see I've added some pages of documentation to a few nodes, I'm going to put up the interactive map today when I get off work, which will let you explore it more.

Why a hardware standard?

I had a comment and a couple emails asking as to what the point of standards are, especially the point of a hardware standard, let us take a short moment to explore this, hopefully after you read this post the entirety of reason will become self evident.

If you take the time to look, everything has standards, the lcd screen in front of you, the hdmi cable connecting your xbox and tv, the cd you put in your drive, everything has a standard, standards promote quality and insure interoperability between components. Lets take a look at how standards between the three branches feed themselves:

Software Standard:

The idea is to creat the worlds first native multitouch operating system based on the linux kernel. The operating system will run and work properly on all certfied hardware.

App Dev Platform:

The most successful and capable software is properitary, adobe, autocad, maya, etc. This is because sales gives the funding to develop them. The applications in turn cause the operating system to be successful. A successful operating system promotes the following of a hardware standard.

Hardware Standard:

The idea is that in the future there will not just be one manufacturer of multitouch tables / boxes, just as there is not one manufacture of mobile devices / touch screens / pc's. By creating a set of standards, if manufacters comply with them then they will be gaurnteed to work with the software. As hardware is purchased it provides funds to further develop the technology and promte growth.

As you can see standards are a good thing, each branch feeds back into the next fostering growth and development.

Thought Map

Here is my thought map for Multitouch Specification Project. I will add to this and continue to develop it. This is pretty much the bare, bare bones skeleton of it. As you can see it it is split up into the 3 main branches I discussed in my last post plus a new administrative branch I will develop to see oversight of all three primary branches.

I used vym to make this, you should be able to find it in most distro's software repositories.

If you want to add onto the thought map or suggest changes let me know, better yet, download it from this link, and email me a copy of your changes.

Update? A BIG UPDATE.

Alright so I have not been up keeping with this blog as well as I would have liked. I am glad though that it seems to have served many people very well! I have received a couple comments and many private messages requesting the live CD's. Here is what is going on with that, and how it ties into everything else:

1. By the time I finalized a live CD that I determined was bug free and ran on all the hardware / virtual hardware that I could get my hands on Ubuntu 10.10 came out.

2. Ubuntu 10.10 broke a few dependencies and other things and is largely untested.

3. Ubuntu itself is a problem in regards to the long term efforts of my project.

4. What is my project?

5. Linux in general on the desktop.

Alright, you can read lots of articles about this, Linux for the most part is awesome when it comes to the server environment, and even with modern desktop distros such as 10.10 Ubuntu, which in my opinion right out of the box is very comparable to even a mac, the desktop Linux has largely failed due to its lack of applications. As good and amazing as some open source software is the simple fact is nothing besides GIMP has come even close to its commercial counterpart, and debatable GIMP still can't touch a well trained Photoshop user. I realize I will probably get a lot of hate mail for this, but it is the truth. Unless you have the millions of dollars , time, and resources to develop something like auto-cad inventor, or Maya, it will never compare.

Back to the point, even with amazing as Ubuntu 10.10 is out of the box, it will and the rest of desktop Linux is failing because of application support. This got me thinking, open source is a great thing for an Operating System but is for the most part counterproductive to software. Also open source doesn't necessarily have to be free of charge either.

Where am I going with this? I think it is fairly self evident that eventually the keyboard and mouse will disappear. The obvious replacement is touch screen which has in the land of mobile computing, largely if not entirely replaced the keypad, just look at iPhone's and Android's. Eventually touch, and naturally multi touch will take over the desktop interface.

As we all know there is really no standard what so ever when it comes to multi touch. There is not even a hardware standard. There is also no real application base out there either. Linux actually has a chance for a massive market gain here. To be the pioneer where others are not yet reaching. All of the building blocks are here they just need to be put together. One must choose an operating system to build these future applications on. I'm not going to get into the details of this one but Linux is the obvious choice here. As far as the hardware is concerned, if you are reading this I am sure you are all to familiar with LLP, FITR, DSI, and every other derivative out there.

I feel like I am getting lost in the sauce here and just rambling, okay to collect all of this into a sensible post:

Here is what I want to do. I want to develop an open hardware solution, a Linux headless TUIO server which will pretty much be an embedded device with the camera attached directly to it; and a Linux distribution with an OpenGL interface with standard application development platform, think Visual Studio and WPF for all Windows developers reading this.

I want to build a team of people that will work on all three of these aspects.

The reason for the open hardware solution, there needs to be a standard for multi touch hardware, so pretty much we will have to pick a technology and stick with it. If you don't understand why there needs to be a standard, I don't even know what to say about that...

The distribution, this is kinda a toughie. If you think about it no current operating system in existence is really suitable for multi touch development. All current operating systems are based around keyboard and mouse input. If Linux were to drop out to the command line and not start the x server on a multi touch table that has no keyboard and mouse it would be pretty hard to recover the system. And no, we are not going to making a new operating system. There are ways for us to work around this problem. As far as basing this new distribution off of a current one, there is not much point, the only reason for that would be package, dependency and application support, but seeing as how all these applications and even the GUI is based around a mouse and keyboard input it is completely useless the multi touch community. All we really need is a bare bones Linux distro with XServer on it and OpenGL support. Remember that a large portion of desktop users spend their time on the internet so the distribution will have to pull all the required pieces to support things such as flash player etc. Other than that, it will be pretty much a blank slate for us to roll with. We need the WPF / Visual Studio style standard because the new technology needs an amazing development platform if companies will ever develop applications for it.

So hardware is never free, but if we get the standard down, we can make a table / slate style design that can be mass manufactured, something that people can pick up at a best buy.

Linux based and open source operating system, free of charge installed on the hardware.

Standardized development platform for companies to develop software for. Software that people will pay for.

I have seen a lot of something in the multi touch community that I really do not like. So much time and argument is being put into creating a gui that is best for or which one is the most correct for multi touch. Something that I have learned from my time in the military is that a timely decision is better than a decision that took a very long time but is more "correct". In the military it costs people their lives. The same principle applies here. A timely decision must be made. That's why who ever joins me on this project, we will come up with a set of standards and stick with it. People will use whatever you give them. They will make it work, and things evolve towards the better over time. Look at windows 95 compared to windows 7. I imagine our first distribution will be more like windows 95 than windows 7. If we keep debating and "researching" critical time will be lost and some one else will come in and take the multi touch reigns.

As many of you can see I am proposing a sort of hybrid thing here. Open standards and an open source operating system, but something that will have the support and backing of privatization.

If you want to join this team, and project send me a comment, email or message, point is contact me some how and let me know. I look forward to hearing from you all some more. and hope you enjoyed my latest post.

Live CD's

Good news everyone, I know I promised live cd's, and I am running a bit behind, but as I mentioned earlier I can't find a cost effective solution for hosting multi-gig iso files. As a result I will be offering live cd's via the mail, I plan on being able to offer them hopefully within the next week.

Another thing I would like to mention, I turned ccv into a background process and turned the gui into a webpage. This would allow you to run ccv on a bare-bones pc and simply connect it to the multitouch pc via Ethernet. You can then control the bare-bones via the web page dished out by apache. I will upload screen shots around the same time I offer the live cd's. It is still a little buggy, especially in the regards of streaming the video, but I really like the idea of a independent tuio server that is platform independent. This tuio server is currently the main focus of my project and I will be putting off the open gl multitouch gui tutorial off for at least another month as there are a few more bugs I need to work out before I offer that as a tutorial.

Back Again!

All right so I have been away for a couple weeks now but I am back!

I see a couple of you have had success and some are having problems. I am going to look at the problems and see if I can't resolve them.

Also here is what is coming up soon:

1. I am going to remaster this into a live-cd so that you can just download the iso and have a working fresh install of ubuntu 10.04 plus CCV.

2. The next tutorial will follow my development of an openGL based windowing system.

This windowing system will incorporate a tuio listner so that everything will be multitouch, even the login after you press the power button on your computer. :)

STEP 6: Compiling CCV on Ubuntu 10.04

Now we are going to compile CCV for Ubuntu 10.04.

Make sure you DO NOT start here. You need to start at the beginning of the blog and follow from there, if you try to do this tutorial BEFORE this one, your ps3 eye will NOT work with CCV.

The following is a modified tutorial based off of x29a's post on the nui group forums.

Enter the following commands:

• sudo apt-get install subversion
• sudo su
• cd /opt
• svn co http://nuicode.svnrepository.com/svn/tbeta/trunk/tbeta/Linux/ ccv-Linux
• cd /opt/ccv-Linux/scripts/ubuntu
• chmod +x *.sh
• ./install_codeblocks.sh
• ./install_dependencies.sh
• apt-get install libpoco-dev
• apt-get install gpp

• cd /opt/ccv-Linux/
• cd apps/addonsExamples/Codeblocks_8_linux/

Now Download the following zip file. It contains the files I modified so that it will build on ubuntu 10.04: http://mt-dev.googlecode.com/files/codeblocks.tar

Unzip it to your Documents folder.

Back in your terminal enter the following command:

• nautilus

Navigate to /opt/ccv-Linux/apps/addonsExamples/Codeblocks_8_linux
and delete the following files:

1. Community Core Vision.cbp
   
2. Community Core Vision.depend
   
3. Community Core Vision.layout
4. Makefile

Now navigate to where you unziped my file. Select and cut the 4 files and paste them in the /opt/ccv-Linux/apps/addonsExamples/Codeblocks_8_linux folder.

Close Nautilus. Back in the terminal enter the following commands:


• cd /opt/ccv-Linux/libs/fmodex/lib
• rm libfmodexp.so libfmodex.so
• ln -s libfmodex-4.22.00.so libfmodex.so
• ln -s libfmodexp-4.22.00.so libfmodexp.so

• cd /opt/ccv-Linux/apps/addonsExamples/Codeblocks_8_linux/bin/libs
• rm libfmodexp.so libfmodex.so
• ln -s libfmodex-4.22.00.so libfmodex.so
• ln -s libfmodexp-4.22.00.so libfmodexp.so
  

• cd /opt/ccv-Linux/
• codeblocks apps/addonsExamples/Codeblocks_8_linux/Community\ Core\ Vision.cbp

Now in codeblocks click Build --> Build.
You may get some errors, just click OK and ignore them.
CCV should now have successfully built. Close Codeblocks and go back to your terminal and enter the following commands:


• cd /opt/ccv-Linux/apps/addonsExamples/Codeblocks_8_linux/bin
• gedit start.sh
  

In gedit copy and paste the following:

#!/bin/sh
echo $(pwd)
export LD_LIBRARY_PATH=$(pwd)/libs/
./Community_Core_Vision



Click File --> Save. Now close gedit. Now enter the following commands into your terminal:


• chmod +x start.sh
• cd data
• gedit config.xml

In the Camera settings change UseCamera to 1. Click File --> Save. Close gedit.

To Start CCV enter the following commands in your terminal:



• cd /opt/ccv-Linux/apps/addonsExamples/Codeblocks_8_linux/bin
• ./start.sh

Congrats! You are now running CCV with the ps3 eye! if the camera feed is not from your ps3 eye but from another camera on your computer don't fear! Go back to the terminal in which your ran /start.sh and scroll up till you see the list of video capture devices, take note of which device your ps3 eye is and change the device number in config.xml to that and restart CCV with start.sh! Thats it! Hope this helps all those out there who want to develop MT on linux with CCV and a ps3 eye. Please comment if my instructions don't work anywhere in this blog.