Je ne suis pas particulièrement un défenseur d’Hadopi, mais je ne compte pas rester les bras croisés pendant que de telles inepties circulent sur Internet. Nos amis nantais (Trident Media Guard) n’auront pas de difficulté a passer au travers de SeedFuck et déterminer aisément les vraies ips.

Si j’était moi même une société nantaise mandatée par l’état pour tracker les ips qui téléchargent et partagent un fichier torrent donné mon mode opératoire serait légèrement différent. Le processus serait simple:

• Connexion au tracker comme étant un client avec 0% du fichier, le tracker envoie une série de peers auquel je peux me connecter. Le tracker va également publier ma propre IP pour permettre aux peers de se connecter à moi.
• J’établis des connexions aux peers et j’attend d’en reçevoir
• Pour chaque peer qui se connecte à moi, ou que je contacte:
  • Je lui demande une partie du fichier qu’elle a et que mes autres peers n’ont pas ou peu (prévu comme ça dans le standard bittorrent)
  • Une fois la partie reçue je compare son checksum à ce qui est indiqué dans le fichier torrent
  • Si ça match, je stock la partie reçue avec l’ip d’origine, et la date/heure. En effet j’ai sous les yeux un flagrant délit de distribution de données sous copyright par une ip “en personne”

Ce mode opératoire rend la détection du client d’analyse Hadopi difficile (se comporte comme un client torrent, et l’usage d’un client id + une ip dynamique changés chaque jour n’aidera pas a la détection) tout en donnant une preuve irréfutable qu’une IP donnée a participé à un acte de piratage.

Seedfuck se contente d’ajouter dans les IPs connues du tracker de nouvelles IPs aléatoires. Cela signifie pas que ces ips vont réellement distribuer le fichier en question. Tout ce que ça fera est de réduire le ratio de peers valides dans la base du tracker, et diminuer la qualité du téléchargement P2P.

Donc je dis bravo à celui qui a imaginé Seedfuck, y’avais pas mieux pour aider l’état !

FAQ

Y’a écrit dans le brevet de TMG qu’ils n’allaient pas faire comme ça.

On peut espérer pour eux qu’ils n’ont pas prévu de rester sur une méthode unique du début à la fin. Si je peux me permettre de le rappeler, le combat entre le bien et le mal est un combat sans fin où chaque côté a l’avantage un moment, et ne l’a plus le moment d’après (à vous de décider quel côté est le bien et lequel est le mal).

De toutes façons le fait d’avoir breveté une méthode (moisie) ne les empêche pas d’utiliser une autre méthode.

Vérifier un checksum pour un morceau de fichier ne permet pas d’être sur a 100% qu’il s’agit bien du même fichier

Oui non hé ho! On parle là d’une IP qui répond au protocole bittorrent, confirme être sur le torrent en question, et qui a fourni des données binaires pour lesquelles le SHA1 correspond exactement. Les chances d’avoir une collision en SHA1 sont extrêmement faibles, pour le moment aucune collision n’a été trouvée, et les chances de collision sont calculées à 1 sur 2^63, ça laisse du temps).

Même si on ne peut effectivement pas être sur a 100% de rien du tout, c’est du 99.99999% avec de toutes façons une IP qui répond au protocole BT (la méthode TMG est plutôt du genre 70%).

PS: Si vous m’en voulez de donner un mode opératoire qui permet de contourner seedfuck ou n’importe quelle autre méthode en générant un cas de flagrant délit (l’ip en question a fourni un bout de fichier qui correspond au checksum du torrent, et donc est valide), essayez plutôt de vous demander pourquoi vous n’y avez pas pensé vous même.

This will create fake IPv4 for IPv6 users based on the 64 first bits of their IP. As most currently exising IPv6 providers are assigning /64 classes to their customers, banning the generated IPv4 effectively bans the whole IPv6.

Also I do generate a 32bits IPv4 from 64bits of IPv6 using XOR. While this means people using IPv6 might share the same generated IPv4 (quite unlikely), it is usually impossible for someone to obtain a different generated IPv4 without access to more than a /64 (I believe only system administrators have this kind of thing).

You might want to store in database generated IPv4 and their IPv6 counterparts to be able to recover a given IPv6 from a blocked IPv4.

This method also allows to block IPs in the same subnet from IPv4 subnets (I don’t know if IPB supports this feature) and recognize people from the same subnet as the start of their generated ip will be the same (however whois information for the given IP will not match the real user’s ip).

To have the generated IPv4, insert this in conf_global.php :

$encoded_ip = inet_pton($_SERVER['REMOTE_ADDR']);
if (strlen($encoded_ip) == 16) {
    $ipv4 = '';
    for($i = 0; $i < 8; $i += 2) $ipv4 .= chr(ord($encoded_ip[$i]) ^ ord($encoded_ip[$i+1]));
    $_SERVER['REMOTE_ADDR'] = inet_ntop($ipv4);
}

And remember to ask your IPB support to have real IPv6 ASAP.

Searching around a bit finally got this error from Chrome:

Unsafe JavaScript attempt to access frame with URL http://bbs.gg.st/index.php?app=core&module=global&section=login from frame with URL http://www.facebook.com/extern/login_status.php?api_key=10e950be918b8f0561e2073c53f2ab8e&extern=0&channel=http%3A%2F%2Fbbs.gg.st%2Finterface%2Ffacebook%2Fxd_receiver.php&locale=en_US. Domains, protocols and ports must match.

On Firefox (and probably other browsers), this works without problem. Just sharing that so other people do not get stuck with the same problem.

Apache has some nice included modules, for example mod_vhost_alias. This module allows someone to configure vhosts by just creating directories however it has some limitations:

• It will cause problems with some other modules like mod_rewrite
• You can’t configure stuff (php options, etc) by host (only with .htaccess files, but you can’t alter all settings)
• It can’t handle variable kinds of domains

I decided to do something better, even with the people on #apache-modules (freenode) saying it’s not possible. It was even no possible to do this cleanly, however looking in apache’s code allowed me to reach my goal without too many problems, but with some really dirty parts.

#define CORE_PRIVATE

To reach my goal I needed to use some functions from Apache2′s core. I just wanted to say that I am really sorry, and won’t do it again (maybe). The functions I used are not meant to be used the way I used them, however I had no choice has there is no publicly available function to change the document root, or to inject configuration directives in the current request.

Anyway don’t do this at home, kids!

ap_get_module_config(…, &core_module)

One of the keys to play with core config dynamically is to fetch it. This is the way to modify ap_document_root. I just return DECLINED after completing my dirty work to let apache think it still has to do its work. Yes this is dirty. But it works.

ap_walk_config()

Ever wanted to do bad things in a per-config context? Now you can. I won’t comment this too much, but I’ll just say that it saved me big time (this one is not part of CORE_PRIVATE, so you can use it freely I guess).

The final step was to make logging easier. I decided to throw all the logging info through a udg socket which is then collected by a daemon, stored locally, and transferred to the logging server at the same time.

Typical whois reply will look like:

   Server Name: MICROSOFT.COM.ZZZZZZ.MORE.DETAILS.AT.WWW.BEYONDWHOIS.COM
   Server Name: MICROSOFT.COM.ZZZZZ.GET.LAID.AT.WWW.SWINGINGCOMMUNITY.COM
   Server Name: MICROSOFT.COM.ZZZZZ.DOWNLOAD.MOVIE.ONLINE.ZML2.COM
   Server Name: MICROSOFT.COM.ZZZOMBIED.AND.HACKED.BY.WWW.WEB-HACK.COM
   Server Name: MICROSOFT.COM.ZZZ.IS.0WNED.AND.HAX0RED.BY.SUB7.NET
   Server Name: MICROSOFT.COM.WILL.LIVE.FOREVER.BECOUSE.UNIXSUCKS.COM
   Server Name: MICROSOFT.COM.WILL.BE.SLAPPED.IN.THE.FACE.BY.MY.BLUE.VEINED.SPANNER.NET

Of course it might looks like the whois server was hacked, that’s what people with bad knowledge of internet would think (hint: almost everything is explained in RFCs).

When you perform a whois lookup, the whois server will usually search domains and return you informations about the domain you requested. However on internet you also have Glue Records which are searchable via whois.

When you perform a whois on, let’s say “microsoft.com”, the whois server will search all records that starts with microsoft.com. Now let’s say the owner of spanner.net created a glue record on microsoft.com.will.be.slapped.in.the.face.by.my.blue.veined.spanner.net, it will match.

Now lots of people did that, so whois records are full of glue records starting with microsoft.com. The only way to limit that is to code a limit in ICANN whois server. So it was decided that only 25 expanded or 50 name-only records would be shown. What happens to the real domain name? It’s also listed as one of the records, usually at the end.

So, nothing was hacked, no whois server was harmed, you just got a bunch of people who are exploiting a specific behaviour of the whois system to make their glue records get listed before the real domains. If you want to appear in microsoft.com you can create a glue record which would look like: microsoft.com.zzzzzzzzzz.uh.did.you.wake.me.up.from.my.sleepdeprivation.com.

Have fun posting stuff on your blog, but stop saying whois servers were hacked when they were not. Anyone who owns a domain name can create glue records, no hacking skills are required to achieve this. You are giving too much credit to guys who just pressed a few options in their registrar’s admin panel (and remember that you can be easily tracked back too). That’s far from what I would call “hacking”, and even not at the “script kiddie” level.

Oh and guyz, it’s been like this for a long time (first time I saw that there was only one record, it was in something like 1998. In the following years more and more records were added to finally reach today’s state). Remember to always verify your sources, even when you got breaking news like “microsoft.com was hacked”.

Finally, I find it amusing to see someone with a MacOS X machine called “bofh” and a green terminal (yay! old school) “discovering” a hack and feeling the urge to report it (and show his green terminal to the world, too). Mac OS X is not a hacker OS, and Apple’s whois client sucks – recent whois client add options to either only receive domain responses, or get expanded responses. Please use a real OS (FreeBSD, Linux Gentoo) or make your own.

Some of my favourite records from a LONG time ago:

MICROSOFT.COM.SHOULD.GIVE.UP.BECAUSE.LINUXISGOD.COM
MICROSOFT.COM.SE.FAIT.HAX0RIZER.PAR.TOUT.LE.ZOY.ORG
MICROSOFT.COM.N-AIME.BILL.QUE.QUAND.IL.N-EST.PAS.NU
MICROSOFT.COM.MUST.STOP.TAKEDRUGS.ORG
MICROSOFT.COM.IS.NOTHING.BUT.A.MONSTER.ORG
MICROSOFT.COM.IS.AT.THE.MERCY.OF.DETRIMENT.ORG
MICROSOFT.COM.FAIT.VRAIMENT.DES.LOGICIELS.A.TROIS.FRANCS.DOUZE.ORG
MICROSOFT.COM.WILL.CRASH.IN.6MN.ORG
MICROSOFT.COM.IS.A.STEAMING.HEAP.OF.FUCKING-BULLSHIT.NET

Of course as I don’t want to bother yet with USB drivers (that’s next in line in the TODO list, but for now we can somewhat survive without), I decided to use our blkdev/bios driver (legacy 16bits driver via BIOS interrupts, using core/x86emu 16bits emulator).
Same with graphics mode, don’t have a Intel driver yet, so we’ll use display/vesa_legacy 16bits driver (which also makes use of core/x86emu 16bits emulator to get graphics adapter’s bios working).

At first, I found out that the new compact bootloader wasn’t working properly. This was fixed by enabling Unreal Mode (a way of tricking the CPU into thinking ds and es segments are really mapped in 32bits mode while still being in 16bits mode).
The next problem I encountered was when the “hard disk” (the usb drive, in fact) was detected, we couldn’t display the size. I tracked this to some weird bug in our current printf() implementation that makes a real system freeze when displaying a float (it works seamlessly on virtual system). Removing the display of detected disk size fixed this problem too, but it’s not a permanent fix.

Once those two points were fixed, everything else worked. I could access our marvelous “shell”, type commands (I issued “lsmod”, “cat sample.txt” and “modp misc/helloworld”) and see that it was good.

My RTL-8139 network controller was detected and module nic/rtl8139 loaded, IPv4 enabled, and most things were working. I believe the next step will be to test the scheduler, optimize it, and start working on the TCP stack. At the same time, a cache system at VFS level and another at block device level should be implemented using a kernel generic cache allocator (that will automatically free physical pages when memory is needed), process switching should make use of CR0.TS to know when we need to switch MMX/SSE2 context, and the keyboard input should be improved.
At the same time, a generic HCI stack is needed for USB, and OHCI/EHCI/UHCI should be implemented (something has been started, but it’s far from complete).

Thanks for everyone who has been providing support to the project, I hope to be able to provide more impressive news soon !

First, port knocking is not meant to be used alone…

Even if you use your daemon’s default port (let’s say port 22 for sshd), port knocking can protect you more than you can even imagine. Let’s take the following setup:

• SSHd running on port 65122
• Connections to port 65122 are replied with “connection refused” (via an icmp target rule)
• In order to “open” port 65122, connection attempts must be made to ports 22448, 44228 and 22884 in this order. Any other order will blacklist the IP attempting to connect for 1 hour
• More than 5 attempts to connect to port 65122 within 20 minutes will result in 1 hour blacklist

Now, if you’re that smart, just try to find your way in without the “passphrase” (which is 22448-44228-22884-65122). If you do too many attempts, you’ll end blacklisted. Let’s say you found out that port 65122 gets you banned when you connect, and have determined that you can make up to 5 attempts in 20 minutes. Let’s also say you know you have to knock exactly 3 ports to be able to connect.
You then have to test 65536^3 = 281474976710656 combinations, and can only test 5 in 20 minutes, that would require 70368744177660 minutes (133882694 years or so).

I can assume no decent system will be up for 133882694 years without any shift into security settings. You can parallelize that with different source IPs, but it will still last too long against people shifting every 3~6 months.

First, the host:

• CPU: 2x Intel Xeon E5405 (2GHz) ; a total of 8 cores
• RAM: 8GB RAM (4x 2GB DDR2 @667Mhz)
• Hard Disk: 2x1TB HDD (RAID 1, 3ware Inc 7xxx/8xxx-series PATA/SATA-RAID) ; total of 1TB usable
• OS: Linux Gentoo 64bits 2008.0 (multilib) with Linux Kernel 2.6.27-gentoo-r2

The test itself will be a 1 million random queries generated by gen-data-queryperf.py with 40% of random domains.

Some words on results

First, I’d like to say that pinetd2 is still under development, some parts are still not implemented (the DNS server is able to act as a DNS server, that’s the important part for me), and also some optimizations weren’t done yet (for example a query will always cause the same SQL statements to be run, I could prepare those).
The fact I’m running SQLite means the SQL server isn’t able to cache results (the db file might be modified by anyone, anytime, however I don’t know the exact internals of SQLite), and I don’t cache anything either.

When I started writing DNSd, I didn’t especially try to go on performances, features were importants, and realtime was too. Many improvements to speed can still be done (I’m thinking “prepared statments” right now, but also caching domains list, etc) and would help to get those numbers closer to ISC BIND.

The fact DNSd is 1/4 the speed of BIND (2531.89 queries/sec instead of 10071.2 queries/seq, my dns server is runnnin at 25.14% the speed of bind) is impressive. I guess we’ll need more tests, with different backends (MySQL is also supported, in theory) and different hosts, but I was supposing the database overhead would be bigger than that (well, SQLite is fast, but I wasn’t expecting that fast).

To tell you the truth, I am surprised by those results, however these are results on a real host, really running domains (like my blog’s domain), which makes me believe those results are the closest I could get from DNSd performances on a real host.

Now, the raw test results with both bind and PHP DNSd, running from the same host (to avoid network latency, and since I got 8 cores with almost no CPU usage as it’s morning in France, it shouldn’t make a big difference).

Other test results with other hardwares are welcome. I’ll try running the same kind of tests on less powerful hardware too, just to see what I get.

ISC BIND 9.6.0-P1

DNS Query Performance Testing Tool
Version: $Id: queryperf.c,v 1.12 2007/09/05 07:36:04 marka Exp $

[Status] Processing input data
[Status] Sending queries (beginning with 91.121.45.45)
[Status] Testing complete

Statistics:

  Parse input file:     once
  Ended due to:         reaching end of file

  Queries sent:         1000000 queries
  Queries completed:    1000000 queries
  Queries lost:         0 queries
  Queries delayed(?):   0 queries

  RTT max:         	0.605333 sec
  RTT min:              0.000035 sec
  RTT average:          0.001974 sec
  RTT std deviation:    0.002666 sec
  RTT out of range:     0 queries

  Percentage completed: 100.00%
  Percentage lost:        0.00%

  Started at:           Wed Feb 18 06:36:21 2009
  Finished at:          Wed Feb 18 06:38:00 2009
  Ran for:              99.293069 seconds

  Queries per second:   10071.196409 qps

PHP DNSd (revision 301) with PHP 5.3.0beta1 and SQLite3 (bundled libsqlite)

DNS Query Performance Testing Tool
Version: $Id: queryperf.c,v 1.12 2007/09/05 07:36:04 marka Exp $

[Status] Processing input data
[Status] Sending queries (beginning with 87.98.170.177)
[Status] Testing complete

Statistics:

  Parse input file:     once
  Ended due to:         reaching end of file

  Queries sent:         1000000 queries
  Queries completed:    1000000 queries
  Queries lost:         0 queries
  Queries delayed(?):   0 queries

  RTT max:         	0.645355 sec
  RTT min:              0.000036 sec
  RTT average:          0.007884 sec
  RTT std deviation:    0.004824 sec
  RTT out of range:     0 queries

  Percentage completed: 100.00%
  Percentage lost:        0.00%

  Started at:           Wed Feb 18 06:38:41 2009
  Finished at:          Wed Feb 18 06:45:16 2009
  Ran for:              394.961920 seconds

  Queries per second:   2531.889657 qps

One (or more) last word(s)

The test method is inspired from a link given by James Collins: “the choices for a nameserver“. While comparison can’t be done between the results there and mine (bind’s result are similar, but as said, there were problems with PowerDNS and anyway we are not running in the same conditions) it still looks like I got some chances into getting closer to be a “real” dns server, with PHP code!

Anyway, remember that “there are two sort of lies, lies and benchmarks.” (source: the previous document).

Now, I guess I have no other choice than writing documentation about “how to install DNSd” and “how to setup a DNSd slave”, that’s going to be fun (if anyone can help, I’d be happy, got a public wiki where the doc can be publied).

OpenOptimus’ website has been defaced because of an exploit in Mantis BugTracker, but mostly because I totally forgot to update this thing for ages.

The site has been taken down, since our little script-kiddie think he’s so smart he could host stuff there without root noticing anything (chmod 0000 owned him).

Ever wondered how to accept both SSL and non-SSL connections on the same port?

Basically, to display such a message, we need to know if the client talking to us is speaking using SSL, or not. This is easily done by reading a few bytes from the stream, but if you do this, starting the crypto using for example stream_socket_enable_crypto() will fail, since OpenSSL won’t find the full client SSL handshake anymore.

PHP offers us a nice solution to fix this, using stream_socket_recvfrom(). By passing option STREAM_PEEK to this function, we can take a peek at the data pending in the socket, and try to determine if that’s SSL or not.

There, we can either try to parse a SSL packet, or instead try to find data we know there should be if the stream is not encrypted.

Doing this for the HTTP protocol is easy. The protocol is text based, and the first word we will get from the client will be something like “GET”, “POST” or “HEAD”. We just check if we got any of those. If we did, we got plain text connection. If we don’t, it means we are probably facing a real openssl client, and we can try to start negociating the link.

I wrote a little example you can download via SVN at http://ookoo.org/svn/snip/https_multi_serv/. Just run “gen_key.sh” in the ssl directory to get a SSL private key, then run the server with PHP. By default it will listen on port 8000, so direct your browser to localhost:8000 with or without SSL (both will work, this is the point of this server).

Feel free to use the code there, I commented it a bit so it should be somewhat helpful, and I officially release it under public domain (or BSD if “public domain” does not legally exists in your country).

By the way it’s also a nice example of async server using stream_select().

Of course it’s not possible to auto-magically determine if the client is talking SSL when he’s not talking first. You could wait for one or two seconds to see if something comes (ie. an SSL handshake) but it’s not really something that could be called good practice… So let’s just keep this for cases where the client talks first.