The Project!

Right,

Over the years I have had to use different sections of authentication protocols (kerberos etc) and user stores (Ldap, Active directory) to fix a features in a pre-exising system, or to add new features.

Also, with all the hype in recent years on single sign on and secure authentication systems, I have always wanted to set up my own fully integrated auth system for all my stuff from scratch, and so, since I’m at sun for a year, in a pretty quiet town, the time is now (why the hell not!)

My basic plan is this:

  • LDAP back end to hold user / service info
  • Kerberos auth for local Linux/solaris boxes (from LDAP back end) (will be using PKI for Kerberos not a shared long term key)
  • Samba (using the LDAP backend) to provide the same LDAP users logon access to windows boxes and services (or www.pgina.org looks promising)
  • To tie my RSA securID keys in (radius, somehow, somewhere) to force all external facing auth systems (read below) to require this extra security.
  • WEB facing net apps to require LDAP login PLUS RSA securID info (via RADIUS i think, although no research has been done on this section at this point)
  • Web Facing OPENVPN Access to also require valid LDAP user (plus hopefully RSA SecurID too)
  • House access points to use WPA2 Enterprise, allowing Radius server to authenticate those too.

And that’s it, The only reason for doing so is ‘I want to’ to see if I can, and to learn the technologies.

A lot more research will be required, and I plan to make a decent effort to keep my findings written down, so by the time I have got it all working on my test system, I can re-create the system with (relative) ease on a real server (and use this time to write a semi-decent howto).

Ok, There are probably going to be a lot of updates relating to this, So instead of clogging the blog; they will all be going in a page (look at the top right of the page for the link) called ‘PROJECTS’ Will always put the newest stuff at the top.

If you want clarification of anything I have written / am doing. Or you feel you could help me with links, comments, etc etc please feel free to post a comment, you don’t have to sign up, or even use your real email / name 🙂

Thanks, and money for counselling after this self subjected ordeal is welcome…
//Matt

Sun StreamStar

While waiting for a few things to install / copy before starting a huuge project (more laters) I thought I would take this time to have a small ramble about the new sun streaming system I was setting up last week;

It’s called the StreamStar, or the sun streaming system (depending on whether you like snazzy marketing names, or more sensible names that better describe the product you are about to buy :P) And it’s suns new solution for massively parallel simultaneous IP Video streaming.

The Jist of it is this:
“The Sun Streaming System’s revolutionary design scales up to provide personalized unicast streaming to every television set – enabling operators to offer new personalized television services faster and more economically, such as customized content per subscriber, targeted advertising, high-definition video streaming and time-shifted television and video on demand with anything, anytime, anywhere television personalized for everyone. The result – operators can increase subscriber revenue at dramatically lower technology costs” – Sun Microsystems (http://www.sun.com/servers/networking/streamingsystem/features.xml)

Roughly translated, This thing is seriously impressive;

  • It has the capability to stream 160,000 2Mbit/second streams to individual users (unicast) SIMULTANEOUSLY! While providing each user the ability to stop / pause / rewind / fast forward etc.
  • The smallest setup comes with 24TB of back end secondary (Hard disk) storage, Enough to store 9,400 Hours of video at 2Mbit.
  • The system can scale up, allowing each component in the system to have one or more systems dedicated to each task, for more processing power if need be (and hot fail over), and allowing the connection of upto 32 x4500 (Back end storage servers) Allowing for a 768TB Content library.

The operation of the system is pretty cool too, On top, there is a web and CLI interface for the administrator to add content, provision new servers into the system etc (More regarding this side of the kit when I’ve had more of a chance to look around)

For now, I just know about the Setup.
The streaming system is made up of a number of components, the storage servers (x4500’s) which have already been discussed. Then there are a number of 1U systems (X4100’s) that manage the health of the system, as well as controlling the system and managing the establishment and control of each users video stream.(Distributed software that all loads via net boot from one of the X4100’s) Last but not least, is the Streamstar Video switch, but more on that later.

The control nodes (X4100’s as mentioned above) co-ordinate (via a seperate, private 1GB/s Switched link) each of the systems..

say a user requests a stream (Most likely through an ISP / cable providers content management system/set top box.. but anyway);
the control nodes (using open standards and protocols such as RTSCP/RTSP) will establish a session with the user, at the same time, will order the back end storage to output the requested content to the streamstar video switch (if not already cached in the switch.. More on this later), which will have been told (by the control nodes also) what to do with the stream, how to packet-ise it up into UDP packets, and the destination IP etc, and it’s doing this for up to 160,000 different streams! at the same time!
The main component that makes this possible (Apart from the impressive compute/data throughput of the other sun systems (ie the X4500)) is the streamstar switch, It is basically a large cache for media bearing UDP packets, that allows control messages from the control nodes which tell it which UDP data packets to assign which IP’s too, It will then send this data out to the operators network via one of it’s 32 (yes.. 32!) 10Gb/s XFP Fiber ports!

(the impressive thing here, is that since the actual client’s session is controlled by one or more of the x4100 control nodes, the streamstar switch only needs to fire out the relevant UDP packets (the actual video stream)..
any control packets from the client are returned to the control nodes that co-ordinate this whole process. Because of this, (apart from the 1Gb/s Copper control link) the streamstar only ever needs to receive (RX) from the x4500 storage array, and transmit (TX) to the network, without worrying about any ACK’s (UDP only remember)

This means that you only use one fiber connection for a connection to an X4500 and the network, as you hook the TX of the X4500 upto the RX of Fiber XFP1 on the streamstar switch (for example) and then the TX of XFP1 to the providers network fabric! This allows for a much better utilization of your fiber network adapters.) … Anyway, back to the streamstar switch itself!

To do this for so many streams, the streamstar switch needs a lot of quick to access memory, and it definitely has that! the system is built in a vertical blade chassis, with each blade having 64 standard Dimm slots (Plus an individual network processor per board to tag the packets etc) This allows for upto 1TB of RAM per streamstar switch! (using 2Gb Dimm’s)

This system is definitely aimed at high uptime in a busy environment too, with each system having multiple redundant PSU’s, easily replaceable (mainly hot swappable) components, and the ease at which the interface allows for failover nodes to be set up.

I have spend nearly a week with this piece of kit, helping some of the designers / engineers who worked on creating the system set it up, and im VERY impressed! sun really seems to be keeping it coming with a great lineup of products! long may it continue!

I’ll let you know more about the management of the system when I have had a peek around it’s interfaces!

//Matt

(I feel I need to say here, just to cover all bases, that this post reflects my own opinions of this product and it’s setup, and what I have written here (apart from the quote directly from the sun website) should in no way be taken as the views of Sun Microsystems inc or its affiliates)

For more reading on this product have a wander over too: http://www.sun.com/servers/networking/streamingsystem/

SATA Cables suck!

Had a great week at work this week! Have been helping install one of Sun’s new products into our lab (More on that in another post) and so this weekend, continuing the techy theme (and since I don’t really have enough money to go out :p) I decided to get on with a few of my projects. The first being update my solaris ZFS NAS box to the newest opensolaris build. (snv_72)

This all went fine, until I had to re-import my ZFS pool into the system, and apparnetly one of the drives was not there. This wasn’t a problem, as it’s a 3 Disk raid (and so the data was still accessible) however, I spent an hour or so trying to work out what was up (missing disks dev path looked quite strange so that lead me down a path of thinking it was a solaris/zfs issue)

But nope! Damn SATA Cables!
I don’t know who allowed the sata cable standard to be released without clips, but he/she should be shot… as now, some cables come with clips, and some without. Some motherboards have sata ports which allow cables with clips to clip into them, whereas some (Mine, don’t) Even different drives may / may not have anywhere for ‘clipped’ sata cables to clip in..

The result, VERY unreliable connections, especially if the drives have been moved around a couple of times, and the sata connections are not as tight as they were. I mean! WTF where they thinking ‘Ahh these cables are nice and small, no-one will care that they can slip out, everyone will just be happier they are faster than IDE!!’

Thats like buying a new BMW that looks better, is faster, and easier to drive… but sometimes the breaks may not work.. ahh well!
</rant>

Anyway, application of superglue to my sata cables has solved this issue, and my NAS is happy again.

My next task is getting solaris running under Xen, so I can use this server box (as it’s pretty powerful) to do other stuff, such as an OpenLDAP server, and a linuxMCE (distributed home media center) server.

Will let you know how it goes.

//Matt

BEER!


20/09/2007

Originally uploaded by TrX07

Just a quick one to test Nokia N95 direct flickr uploading! And then the wordpress XMLRPC API that allows you to add photo’s to your blog direct from flickr!

I think with this i’m going to start adding more photo’s to blogs (now it requires so little effort!

So to start off, here’s a selection of beer on sale at morrisons! 😛

-Enjoy
//Matt

Comments Open

Just a note to let you know that I have installed some decent comment spam filtering for wordpress, which seems to be doing the job!

So comments are now open once more (if you have commented before and it didnt get up onto the site, please dont let this put you off, i probably just missed it in my weekly purges of thousands and thousdands of spam comments)

//Matt

Monthly Roundup

I MUST promise myself to update this more than I am doing at the moment!

Another month has passed, and Ive done quite a lot, the majority of which will dissapear from my memory as soon as I start trying to write about it.. *Wait for it…. there it goes*

This month things have started to heat up at work, As all but one of last years interns have now left (and lamsey (the last intern) is here mainly to continue some in house development of our day to day lab management tools) I, along with 2007’s other interns are picking up the tickets, and dealing with the majority of problems on our own, which feels pretty good. It’s also amazing how much we are helping each other, as some of us will have spent a day over one peice of missbehaving hardware, and then the next week can provide knowlege to another person in the same boat.

On a social side,

We had a BBQ sometime last month (told you i’d forget things when i started writing) as a late housewarming and early leaving do for the 2006 interns. Lamsey, James, Kim and Charlotte came (from the 2006 bunch) and most of 2007’s guys were there (except fraiser and mike c)

Much alcomahol was enjoyed, the music was loud, and the party went on till 4 with much merryness! (also, the specticle of a drunken Robin trying to tell his girlfriend the differences between Vi and Vim :P)

A few weeks later (beginning of this month when we had all been paid) we headed out to a club in farnborough called quarantine. Was a pretty good night, with most of us not remembering the later hours (and it was also nice to get out of the house, as fleet is a pretty quiet place, and no-one seems to be up for going out that much…)

James crashed on our couch that night, and spent most of the saturday in the same position recovering!

On the techy/geeky/whatever side, I moved my routing box to gentoo this month, and got all the services back up and running. It also gave me a chance to neaten up my firewall /routing rules and FINALLY work out a way to fix a reverse path routing problem I was having when performing static routing + DNAT up a openVPN tunneled interface.

(The jist of it was a packets journey through IPTables looks up the destination interface in the kernel routing table too early for what i wanted to do, before it has been un-dnatted, and so rules to send matching data up a certain tunnel never got matched)

I now have a much more customisable platform for starting my next projects (QoS, Ldap auth etc)

Mike also got himself a VOIP Deskphone, supporting the IAX2 protocol (a ‘better-than-sip’ SIP protocol that only requires one UDP Port) and a voiptalk.org account, we linked his phone upto my asterisk pbx, and now calls from his phone use his voiptalk trunk, and mine use mine. Icoming calls also work perfectly, with excellent call quality even if we are both on the phone at the same time. (Using usenet dosn’t help call quality much though! I think some QoS traffic shaping may be on the cards)

Talking of VOIP, this brings me neatly onto my new toy! I finally got fed up with my windows smartphone, chucked it out (well.. look the sim out and permenantly installed it in my car as a tomtom) and got myself an N95 on a shiney new T-Mobile contract (winged so much about how crap my windows smartphone was that they let me end my last contract nearly three months early)

Got the new phone two days later, and I have to say, unless something MAJOR changes in the phone industry, i’m ONLY buying nokia’s from now on. They just always hit the nail on the head, nice phones, decent weight, feel well built, sensible connectors, good interface (everything is really nice and integrated) and it JUST WORKS! even the software, for backing up my phone and all the settings (even what shortcuts i have assigned to my softkeys on the idle screen!) syncronising my messages (+ sending and receiving messages from the PC)

If you don’t yet get the picture, I am VERY impressed with the phone. UMTS 3.5G (HSDPA), WIFI, GPS, Bluetooth (and seems to have a sensible bluetooth stack that scans pretty fast too), decent size screen, 5 megapixel camera, nicley laid out buttons, cool sideways slide gally viewer thing… and TV OUT!! yup.. comes with a nifty little cable (that plugs into the standard size 31/2mm headphone jack and gives composite out plus audio onto a PAL or NTSC TV (looks pretty damn good for watching a movie (has a 2GB microSD card) or showing photo’s you have taken to your mates)

Also the software on the phone is really nice too, its still running on symbian S60, but with cool things like a SIP VOIP client, properly integrated (so everywhere there is a number and you can click call… you can just as easily click options>internet call instead to route through to your SIP provider)

I hooked it upto my internal WIFI and asterisk PBX, works perfectly, can make and receive calls via wifi with excellent quality and hardly any latency. I then tried setting the phone up to connect to my asterisk PBX via the internet (lots of port forwarding later) and I still could not get the SIP protocol to live with my NAT (it really, really dosnt like nat) I was getting the RTCTP packets, and so my internal phone rang when I called it from my n95 (via t-mobile web and walk) however I got no audio.

I even tried the new (beta) SIP_conntrack and SIP_nat modules which should dynamically sort all the SIP Nat problems out for me by changing the IP info stored in the realtime audio control section of the egress packets, but to no avail (I am still planning to have another bash at this sometime. I was going to cheat and VPN in. however no-one has made an openvpn client for S60 phones (I’m already missing it as a feature 🙁 )

Another feature I found out about today, is that it has built in functionality in the photo gallery to ‘send to flickr’ as well as ‘send via mms etc etc) which I tested out, and it worked great! both with a wifi point I was near the first time, and through UMTS 3G the next. (May have to get flickr tied into this blog now it uploads pics so easily)

The only thing I have not really played with is the GPS yet. I am waiting for tomtom hackers to get the internal GPS chip working with the symbian tomtom software.

There is other stuff I want to ramble about (such as plans for the next solaris ZFS fileserver running xen to virtualise some other stuff) but I think that requires a little more research first 🙂

That should have made you leave one page open long enough to make your browser feel loved again! Night!

//Matt