You probably know SSH at least by its name. It’s a of secure telnet replacement which also allows many other things such as port forwarding, remote file management (with sftp) and more.

With PHP I could write a fully working SSH server in only 3 days. Of course I didn’t implement every single extension there is to SSH, but I’ve implemented:

• SSH2 protocol only (no SSH1, anyway who uses that anymore?)
• Encryption protocols: aes128-cbc,blowfish-cbc,serpent256-cbc,cast128-cbc,3des-cbc (via mcrypt)
• Message digests: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96 (via hash)
• No compression as I cannot easily keep a compression context active (the gzip extension in php is missing a way to create a compression context)
• Password and public key (ssh-dss and ssh-rsa) identification
• Ability to program an interactive shell in PHP (there are send and recv functions in a separate class, anyone can have some fun and write something out of that. Should be possible to make a wrapper to communicate with a shell launched with proc_open)
• Support for multiple channels
• SFTP subsystem
• Can be easily extended to add support for custom channels or re-use the ssh protocol for something else

My goal when writing this was to provide a replacement for the FTP protocol for the customers of my hosting service. FTP has many drawbacks, including no encryption (you can with ftps or ftpes) and the way ftp transmits data (another connection has to be opened on a different port, leading most of the time to some problems for people behind a NAT and/or firewalled servers).

With this ssh server supporting sftp, I finally got the replacement I was looking for. Of course it uses more CPU than a C ssh server (about 3 times more) but the difference isn’t that big. Next steps will include fork()’ing to open channels (will allow the SFTP server to chroot) and maybe support for some SSH extensions.

To implement the SSH protocol the following PHP extensions were used:

• OpenSSL: used to generate secure bits when negociating the key, and used to generate the host signature on connection. I was hoping to use openssl_verify() to verify the key used when logging in, but I couldn’t manage to convert the ssh-rsa key to something openssl would understand, so I re-implemented signature verification with gmp.
• MCrypt: The ssh protocol is encrypted (usually with something like AES128). mcrypt has the required functions to handle encryption in block mode
• Hash: each packet transmitted over SSH is optionally signed with a HMAC signature. In order to generate and verify those signatures I used hash_hmac()
• And finally the most important: GMP. As I was missing some functions to properly handle the initial Diffie-Hellman key exchange (and later to implement publickey authentication) I had to re-implement those in PHP. Of course working with 1024 bits integers is not something we can use the native int type for. GMP (and bc) allows such calculations (and I used them). I was missing the ability in gmp to read from/convert to binary values, so I had to add the use of bin2hex() and pack(‘H*’, …) to be able to work with binary values easily. GMP computations are only used when negociating keys (the ssh rfc recommands doing this once an hour, or every gigabyte of data transmitted) or when using the publickey authentification.

What did I create a ssh server for? The same thing I created a DNS server for fun and for KalyHost. In order to provide services updated in realtime I wrote a database-backed dns server a while ago, and now a ssh server (which can be database-backed too by extending the “Base” class).

The sourcecode can be downloaded from the SVN: http://ookoo.org/svn/pinetd2/trunk/code/classes/Daemon/SSHd/ this depends on pinetd2, a framework I wrote which allows to easily create daemons in PHP, and which I already used to create various things (FTP, Mail server, etc).

People willing to help on pinetd2 (code and/or documentation) are welcome. If you do not mind being called crazy because you make something else than webpages in PHP, you can contact me by mail or on IRC (or by leaving a comment on this post too if you wish to).

Current SVN release includes many fixes and improvements, including for SimpleDNSd, the DNS daemon written in PHP.

This includes:

• Support for delegation-only zones: it is now possible to handle TLDs via SimpleDNSd. I did a test by adding “free” domain names to the KalyHost service. Those domains can be ordered for free, and a webinterface is made available to control the domain DNS, allowing you to test SimpleDNSd and see how easily changes are done in realtime.
• Support for PHP new requested feature (PHP bugreport #51295): queries to the DNS daemon were failing or returning wrong data randomly because of this bug. It took me a while to point this out as it was rather random. Basically current PHP implementation of SQLite3 has no “busy timeout”, meaning requests will fail immediatly if database is busy.
  I had to add a busyTimeout() method in SQLite3 (similar to the one already existing for the old sqlite PHP extension) and use it. This means we’ll have to wait for this patch to be added to a current PHP release before pinetd2 can be released as stable.
• PZC: “Progressive Zone Change”. This is one feature no other DNS daemon has (or maybe they do, I don’t know). This feature allows to schedule change of a domain to a new zone. When the scheduled time comes closer, the DNSd will send expiration time smaller and smaller to make records expire on the time the zone will change.

A bit more about PZC:
Let’s say we have domain “example.com” pointing to zone A. Calling API method domainPzc(‘example.com’, ‘B’, time()+86400); will make domain example.com pointing to zone B in 24 hours. In the meantime, no returned record will expire after the scheduled time for zone change: any record obtained 15 seconds before zonechange will be marked to expire in 15 seconds.
This features allow a really precise control of “DNS Propagation”: you decide exactly when zone change will happen. Note that if you have a record in your zone expiring in 3 days, you shouldn’t schedule zone changes less than 3 days before effective date, or it might not have the expected behaviour.

As far as I know, no other DNS server support such a feature allowing to switch to a different zone with full control of when it will “propagate”.

(I know some resolvers out there will not follow expiration times given by the authoritative DNS server, however I like to think those are only a minority, and that PZC will give the expected behaviour for almost everyone)

The new launcher is the first step toward the ability to dynamically start and stop subprocesses without restarting the whole daemon. The old “start.sh” has been replaced with a “daemonctl” tool, like in the old pinetd. This allows to start pinetd2 in the background (if forking is enabled), see processes status, and stop/restart the daemon directly. However since daemon can now be started in background, another feature was needed, which is…

Logging to a file

Of course, the whole system was made to centralize logs. Each child transmits logs via IPC to its parent, and the main launcher display logs on screen. It wasn’t difficult from there to write logs to a file. This means that even a FTP client chrooted somewhere can still send log events to its parent and get them written to the main logfile.

What’s next?

Now, I need to make those two elements less hack-like (ie. add some config options, add ability to log to syslog, etc), then I’ll have to finalize PMaild’s IMAP daemon thanks to the fact I added mailparse to the required elements in the PHP distribution. MailParse will handle the difficult mime-decoding part, and will make the next steps much easier. Parsed data will be cached in database (one new table per domain), and this should finally makes PMaild usable in imap-based webmails like Horde IMP.

All efforts will be put on pinetd2 with its new framework, and improvements required to support transports.

A new pinetd branch will eventually be launched with a complete rewrite of the IPC subsystem (which has become too complex with the different new features added along the line). New branch will most likely be named 2.1.

Of course normal frameworks are a no-go. Using someone else’s framework will make your world slightly better, but until you create your own full framework, you won’t understand what I mean.

The next step is to build applications with your framework. The kind of applications that will change the world…

The rest is up to you…

<m:human-readable errcode="200030">
Could not open the requested SVN filesystem
</m:human-readable>

After some searches I found someone who had the same problem, and fixed it. Basically the problem is due to the SQLite version embed into PHP 5.2.x. PHP includes SQLite 3.3.7 while SVN will usually depend on 3.6.15 (or whatever you have on your system).

To fix this problem you have various solutions:

• Disable PHP as did our japanese friend
• Upgrade SQLite version provided with php, or tell php to use system’s SQLite with –with-sqlite=/usr
• Create your svn repository with –pre-1.6-compatible

Hope this article will help all those who, like me, got the error 200030 with SVN without understanding much where it comes from.

Yahoo! and Microsoft announced an agreement that will improve the Web search experience for users and advertisers, and deliver sustained innovation to the industry. In simple terms, Microsoft will now power Yahoo! search while Yahoo! will become the exclusive worldwide relationship sales force for both companies’ premium search advertisers.

This sounds rather awful, and reply from Rasmus was quick.

Of course I immediatly tried to confirm this on IRC…

[21:20:51] <MagicalTu> http://twitter.com/rasmus/status/2908526145 <- wtf ?
[21:21:21] <Rasmus> See the Microsoft-Yahoo press release

So now, we’ll see how it evolves, but I guess this press conference will change a lot of things, including in PHP’s world.

Basically, you’ll need subversion, gcc, make and most prerequisites for PHP (ICU, etc). This supposed you already have some knowledge in system administration.

The first step will be to fetch the latest SVN code somewhere in your Linux box (I usually put that in /usr/local).

$ cd /usr/local
$ svn co http://ookoo.org/svn/pinetd2/trunk pinetd2
(checkout lines)
$ cd pinetd2/php
$ ./do_php.sh
(will download & compile PHP 5.3.0, and maybe complain about missing stuff,
 just install whatever is missing, if you need help, post the last error
 lines here or contact me)
$ cd ..
$ ./start.sh
Please edit config.xml and remove the line containing this text

Arrived at this point, a file config.xml has been created and needs to be edited. I added comments inside it to help you. You’ll need to do a few things:

1. Remove the <RemoveMe> tag, and edit the <Name> tag to include your machine’s name.
2. Change the storage engine to use SQLite3. PInetd’s MySQL driver is not compatible yet with DNSd. You can copy the line from the example, but remember that if your zone file is in /tmp, it might get erased at startup (depends on your linux distribution).
3. There is an empty <DNSd> tag, fill it with the second choice in the comment (the <PeersArray> one). Change the Signature, peer name, etc to fit your needs for the Type=”control” line : this is the definition of “who will be able to create/remove records on this DNS server”.
4. At the end of the file, remove all processes that do not have Daemon=”DNSd”. There sould only be 3 remaining processes.
   If you are not root, set PortOffset to an arbitrary value greater than 1024, like for example 10000.

Once you’ve reached this point, you should be ready to go. Try starting the daemon.

[2009-02-19 10:10:33:30945] DEBUG: pinetd v2.0.0alpha running on...
[2009-02-19 10:10:33:30945] WARN: SUID security level is defined...
[2009-02-19 10:10:33:30945] WARN: Warning: Chroot security level...
[2009-02-19 10:10:33:30945] DEBUG: My name: localhost
[2009-02-19 10:10:33:30947] INFO: Loading Daemon\DNSd\UDP on port 10053, bound to ip 127.0.0.1
[2009-02-19 10:10:33:30948] INFO: Loading Daemon\DNSd\TCP on port 10053, bound to ip 127.0.0.1
[2009-02-19 10:10:33:30949] INFO: Loading process Daemon\DNSd\Process

As you can see, I am not root, so PInetd will complain about the impossibility of chroot()ing or setuid()ing, however it’s just for testing, so we don’t really care.

At this point, if you configured the DNS daemon like me with a PortOffset of 10000, with a peer named “MyPeer” and a secret of “qwerty”, running “dnsd_test.php” in the “test” directory will create an “example.com” domain.

$ php dnsd_test.php
Connected to localhost

Now, you can test it:

$ dig +short -p10053 @localhost example.com
127.0.0.1

Of course the PHP DNS Daemon is not completed yet (fixed a potential denial of service yesterday) and probably still have many bugs, so I strongly advice against using it on any production system yet.
Yes, I am using it for this blog, and for another ~200 domains, to find bugs and make the solution more stable, however if you wish to contribute by testing on a production system too, make sure you are ready to have all your websites becoming down and your server taking fire.

The next step for you is to look in test directory at the “dnsd_test.php” script, and make your own pages using DNSd. You do not need to use PHP 5.3.0 to use this class, so you can basically use it anywhere. PHP 5.3.0 is only required for the DNS daemon itself.

If you have any question, feel free to leave a comment, I’ll try to complete this article as problems are raised by people who use this.

First, the host:

• CPU: 2x Intel Xeon E5405 (2GHz) ; a total of 8 cores
• RAM: 8GB RAM (4x 2GB DDR2 @667Mhz)
• Hard Disk: 2x1TB HDD (RAID 1, 3ware Inc 7xxx/8xxx-series PATA/SATA-RAID) ; total of 1TB usable
• OS: Linux Gentoo 64bits 2008.0 (multilib) with Linux Kernel 2.6.27-gentoo-r2

The test itself will be a 1 million random queries generated by gen-data-queryperf.py with 40% of random domains.

Some words on results

First, I’d like to say that pinetd2 is still under development, some parts are still not implemented (the DNS server is able to act as a DNS server, that’s the important part for me), and also some optimizations weren’t done yet (for example a query will always cause the same SQL statements to be run, I could prepare those).
The fact I’m running SQLite means the SQL server isn’t able to cache results (the db file might be modified by anyone, anytime, however I don’t know the exact internals of SQLite), and I don’t cache anything either.

When I started writing DNSd, I didn’t especially try to go on performances, features were importants, and realtime was too. Many improvements to speed can still be done (I’m thinking “prepared statments” right now, but also caching domains list, etc) and would help to get those numbers closer to ISC BIND.

The fact DNSd is 1/4 the speed of BIND (2531.89 queries/sec instead of 10071.2 queries/seq, my dns server is runnnin at 25.14% the speed of bind) is impressive. I guess we’ll need more tests, with different backends (MySQL is also supported, in theory) and different hosts, but I was supposing the database overhead would be bigger than that (well, SQLite is fast, but I wasn’t expecting that fast).

To tell you the truth, I am surprised by those results, however these are results on a real host, really running domains (like my blog’s domain), which makes me believe those results are the closest I could get from DNSd performances on a real host.

Now, the raw test results with both bind and PHP DNSd, running from the same host (to avoid network latency, and since I got 8 cores with almost no CPU usage as it’s morning in France, it shouldn’t make a big difference).

Other test results with other hardwares are welcome. I’ll try running the same kind of tests on less powerful hardware too, just to see what I get.

ISC BIND 9.6.0-P1

DNS Query Performance Testing Tool
Version: $Id: queryperf.c,v 1.12 2007/09/05 07:36:04 marka Exp $

[Status] Processing input data
[Status] Sending queries (beginning with 91.121.45.45)
[Status] Testing complete

Statistics:

  Parse input file:     once
  Ended due to:         reaching end of file

  Queries sent:         1000000 queries
  Queries completed:    1000000 queries
  Queries lost:         0 queries
  Queries delayed(?):   0 queries

  RTT max:         	0.605333 sec
  RTT min:              0.000035 sec
  RTT average:          0.001974 sec
  RTT std deviation:    0.002666 sec
  RTT out of range:     0 queries

  Percentage completed: 100.00%
  Percentage lost:        0.00%

  Started at:           Wed Feb 18 06:36:21 2009
  Finished at:          Wed Feb 18 06:38:00 2009
  Ran for:              99.293069 seconds

  Queries per second:   10071.196409 qps

PHP DNSd (revision 301) with PHP 5.3.0beta1 and SQLite3 (bundled libsqlite)

DNS Query Performance Testing Tool
Version: $Id: queryperf.c,v 1.12 2007/09/05 07:36:04 marka Exp $

[Status] Processing input data
[Status] Sending queries (beginning with 87.98.170.177)
[Status] Testing complete

Statistics:

  Parse input file:     once
  Ended due to:         reaching end of file

  Queries sent:         1000000 queries
  Queries completed:    1000000 queries
  Queries lost:         0 queries
  Queries delayed(?):   0 queries

  RTT max:         	0.645355 sec
  RTT min:              0.000036 sec
  RTT average:          0.007884 sec
  RTT std deviation:    0.004824 sec
  RTT out of range:     0 queries

  Percentage completed: 100.00%
  Percentage lost:        0.00%

  Started at:           Wed Feb 18 06:38:41 2009
  Finished at:          Wed Feb 18 06:45:16 2009
  Ran for:              394.961920 seconds

  Queries per second:   2531.889657 qps

One (or more) last word(s)

The test method is inspired from a link given by James Collins: “the choices for a nameserver“. While comparison can’t be done between the results there and mine (bind’s result are similar, but as said, there were problems with PowerDNS and anyway we are not running in the same conditions) it still looks like I got some chances into getting closer to be a “real” dns server, with PHP code!

Anyway, remember that “there are two sort of lies, lies and benchmarks.” (source: the previous document).

Now, I guess I have no other choice than writing documentation about “how to install DNSd” and “how to setup a DNSd slave”, that’s going to be fun (if anyone can help, I’d be happy, got a public wiki where the doc can be publied).

[18:25:06] <Derick> MT`AwAy: you’re mad

Now that it’s said, let’s move on ; if you want to tell me I’m crazy, you can post it in a comment here, it makes me happy. I had some reasons to dislike bind9 which finally made me write my own DNS daemon, and I’ll explain that here.

My need was to have a stable dynamic DNS server working in most environnements, with an easy to configure master/slave relationship (with realtime synchronisation), and a way to change records instantly from PHP…

For those who already played with bind9, there’s a feature called dlz which basically allows to ask bind to get data from an SQL server. I could just configure another MySQL per slave, and put replication there, but it’s not that fast and I had some stability issues (both with MySQL replication, and with bind crashing in some weird cases).

So, instead of trying to fix bind9′s code (which would also include fixing MySQL replication – or trying newer MySQL’s row-based replication) or searching for another solution (there are zillions of dns servers around there, but they all have features I’ll never need), I decided to write my own DNS server (built on top of PInetd, my PHP networking framework for PHP 5.3.0), with only the features I needed.

So here are some of the features of this DNS daemon:

• Supports RFC 1035, and some others too (IPv6 AAAA records, and DNS OPT, with the goal of supporting DNSSEC at some point).
• Does NOT support AXFR nor IXFR, I have no need for the standard zone transfer protocol
• Data can be updated realtime with a provided client class, connecting to the server via a shared secret authentication (client computes checksum of his name, timestamp and secret, server replies with the same kind of data). Once authentication is finished data is sent cleartext, but adding SSL encoding wouldn’t be that hard (just need to create a “STARTTLS” function, that’s on the TODO list).
• Slave stays connected to master (keepalive packet sent every 15 minutes), and gets updates realtime.
• On first connection, slave will get all zones/domains/records from the master. If it gets disconnected later and reconnect, it will search for his last update, and will ask only for newer data from the master.
• Slave synchronisation is done in a separate process, meaning that even when processing a lot of updates, service is available.

As a test, I ran AFNIC’s zonecheck on a virtual zone I created on the server, and it works!

dig @dyndns1.ookoo.org version.dnsd ch txt

If you want to test this, I got a test domain with a record creation page, however I won’t post it here (or it might get abused) so if you want to test, feel free to contact me.

And finally the conclusion is simple: php can do anything you want to do, and even things you never wanted to.

PS: I’m looking for someone to look after PInetd‘s website, like removing this lipsum news, etc… (I don’t have time for that).