Planet FreeBSD

Edwin Groothuis: Migrating from i386 to amd64 - ldd(1) on 32 bit objects

Fri, 04 Jul 2008 00:00:10 +0000

In the last couple of months I have migrated left-over applications on old i386 servers running FreeBSD 4.something to jails on amd64 servers running FreeBSD 6.3. It works fine, it works like a charm, thanks to the 32 bit compatibility and the misc/compat4x port. I could just copy everything in /usr/local/ to the new jail, run ldconfig -32 /usr/local/lib and everything started without problems.

There was only an issue with the ldd(1) command in the base-system:

[/] root@ed-exigent>ldd `which httpd`
ldd: /usr/local/sbin/httpd: can't read program header
ldd: /usr/local/sbin/httpd: not a dynamic executable

The man-page of rtld(1) revealed that it could work:

[/] root@ed-exigent>LD_32_TRACE_LOADED_OBJECTS=1 `which httpd`
libm.so.4 => /lib32//libm.so.4 (0x280c8000)
libaprutil-1.so.2 => /usr/local/lib/libaprutil-1.so.2 (0x280de000)
libexpat.so.6 => /usr/local/lib/libexpat.so.6 (0x280f2000)
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x28110000)
libapr-1.so.2 => /usr/local/lib/libapr-1.so.2 (0x281fd000)
libcrypt.so.3 => /lib32//libcrypt.so.3 (0x2821d000)
libpthread.so.2 => not found (0x0)
libc.so.6 => /lib32//libc.so.6 (0x28235000)
libpthread.so.2 => /usr/lib32/libpthread.so.2 (0x2830d000)

So ldd(1) had to be educated about the differences in the ELF header for 32 bit and 64 bit ELF objects. Once that was done, no problem at all anymore!

Patches are available at bin/124906, have been commited to HEAD and will be MFCed in a week.

Mark Linimon: Bugbusting: good news and bad news

linimon — Thu, 03 Jul 2008 11:40:11 +0000

There is good news, and bad news, on the bugbusting front.

I think we have made good progress on the state of the PR database recently. In particular, with the help of our recent volunteers attractedby the Bugathons, we have been able to filter the backlog, identifying the majority of the PRs by what manpage they are correlated with (for kern/ PRs, primarily sections 3, 4, and 8 via ‘tags’; for bin/ PRs, primarily sections 1 and 8). There are now prototypes of HTML reports that reports corresponding to each of these. (They are not yet interactive; they run once per day on freefall). Also, all the PRs with patches are now tagged by ‘[patch]’, and there is an HTML report for those, too. There are over a thousand of these, representing a significant percentage of the database.

In addition, we have added pages to list a highly-filtered set of a few dozen PRs that we think are ready-to-go, and another set of PRs that are the most-commonly-seen set.

Further, by request, we have added some scripts on freefall to allow a committer to pick a random PR to look at.

(I consider all of this work to be effective prototypes of pages that we would need to customize for any PR system. I have seen nothing yet that supports exactly what FreeBSD needs, out-of-the-box.)

(fwiw, most of this has been by suggestion, from either our new bugbusters, or src committers.)

Finally, many of the “I do not know how to make xyz work” PRs have been either closed, or are being handled by the new volunteers.

I have advertised all of this work on the wiki (including the relevant hrefs) on http://wiki.freebsd.org/BugBusting/Resources, and by announcements on the freebsd-bugbusters IRC and mailing lists, as well as selected cross-posting to other mailing lists.

So, what’s the problem?

Well …

This has resulted in only a handful of src commits.

Apparently I haven’t figured out how to get the src committers sufficiently interested to investigate these pages. I have gotten very little emailed feedback, and few committers showed up for the last bugathon.

So, the challenge is to keep our bugbusters interested in doing the triage work. IMHO, to keep the momentum that has been built up, we need to start getting some commits in.

Unfortunately, I do not seem to have the right ideas in place to make that happen. This is very frustrating.

I would be interested in getting suggestions on where to go from here. I am clearly missing something.

Edwin Groothuis: CVSROOT-ports/modules re-animated

Thu, 03 Jul 2008 10:40:00 +0000

When it comes to the FreeBSD Ports collection, the CVS repository and the GNATS bug tracking system, people not always understand how its can be made easier.

For example: Did you know that the URL http://bugs.freebsd.org/12345 would bring you automatically to the PR with number 12345? And did you know that if you want to checkout the directory /ports/x11-wm/fvwm95, all you have to do it cvs co fvwm95 (For an easy Mozilla Firefox and Seamonkey (and probably other browsers) sidebar, visit this page)

So, what is the story about that cvs co fvwm95. The magic for that is stored in the file CVSROOT-ports/modules, it contains a list of all ports and their directories. The file is updated every time a port gets commited, removed or changed from location. At least that is the theory. Adding is done manually via the addport script, but removing and changing locations has been always a manual task.

Recently the caretaker of this file had enough of people whinching when he said they needed to update that file when it changed. So he said that support for it would cease. But but but... That would break at least one of my ways to do things easy!

I can do only one thing now, take control over it myself! So I proposed it to portmgr@, who approved it. The second thing to make is a script which mangles /usr/ports/INDEX to generate a modules file (in the shape I like it).

All that is done now, and the CVSROOT-ports/modules file is now updated once per day. See the CVS webinterface for the last update!

Anders Nore: Vacation

andenore — Wed, 02 Jul 2008 18:37:43 +0000

Today I’m going on a vacation to Greece for one week. I’ll be back next Thursday (10th). I’ve polished my project pkg_improved so I think that people could test it out to see if it works. There’s not a whole lot of change, just some speed gains here and there, you can check the CHANGES file posted for more details. You could also make a diff with /usr/src/usr.sbin/pkg_install/ but you will get a whole lot of syntax changes =(

If you want to test it, then just compile and install (NB! this will replace your normal pkg_* tools). You could always get back your original pkg_* tools by compiling and installing from /usr/src/usr.sbin/pkg_install . If you don’t want to replace your pkg_* tools you should just compile it and rename the executables (NB! You will not get proper behaviour from ports unless you change some PKG_* variables in /usr/ports/Mk/bsd.port.mk (see PKG_CMD). The first thing you should do after install is to run pkg_convert to cache the existing flat database, after that you don’t have to run it again unless you use the original pkg_* programs.

There’s some ideas that I’ve been looking into like adding date’s to installed packages (perhaps a @comment in +CONTENTS), recovering +REQUIRED_BY in pkg_add (after e.g. pkg_delete -f zip; pkg_add zip). These should probably be posted to a mailing list, but I’ll wait until I get home from Greece to post them.

pkg_improved: http://home.no.net/andenore/pkg_improved.tar.gz

(please e-mail me if you find bug’s and the like)

Ryan French: Weekly Report - 30/06/08

Ryan French — Mon, 30 Jun 2008 01:52:29 +0000

Real Life really doesnt like me working on this project. Thanks to exam time at university I was have to cover a few shifts for other students this past week, and ended up doing 6 days in a row. Thankfully I have some holiday time coming up starting this Thursday, and while I’ll be away for both weekends I’m on holiday I’ll have a full week to work on MPLS from the moment I get up to the moment I go to sleep, without any distractions.

Accomplished Last Week:

- Worked on my FreeBSD wiki pages.

- Went over OpenBSD/Ayame code again to try and get a bit more understanding on how it works.

Plans for Next Week:

- Port over the protocol description file mpls_proto.c, then begin working on porting over code for simple sending and receiving of MPLS packets.

- Once sending and receiving is working, look at implementing static label switching, then working towards dynamic label switching off a routing table.

Gleb Kurtsou: filtering on bridge

gleb — Sun, 29 Jun 2008 19:48:24 +0000

There used to be a flaw in using ipfw on bridge interface. It’s impossible to distinguish incoming packets on member interface from incoming packets on bridge itself. For example consider two rules:
add 1 allow ip from any to any in recv bridge add 2 allow ip from any to any in recv member

First rule will never match. The logic is ok here (if you are aware of ipfw’s handling of interface options). But what do you expect if you disable filtering on member interfaces and perform filtering on bridge only. You expect rule 1 to match all incoming packets on bridge. It gets extremely annoying when using stateful filtering.

First time I came across this issue several years ago. But didn’t figure out how to fix it. At that time I’ve decided to switch to pf.

Actually ipfw is the only firewall that allow rules like
allow ip from any to any out recv if1 xmit if2
Such tricks are possible because ipfw gets input interface from mbuf of a packet. pf for example relies on pfil to provide interface.

I’ve added a hack into if_bridge to work around it. It contradicts traditional ipfw behaviour a little but seems to be much more useful. I think patches are useful enough and can be commited into FreeBSD:

perforce.freebsd.org/changeView.cgi?CH=143921
perforce.freebsd.org/changeView.cgi?CH=144238

Philip Paeps: Patches everywhere

Sun, 29 Jun 2008 16:54:00 +0000

It is amazing how patches have a tendency to pile up on laptop disks. While MFC'ing a silly change this afternoon, I accidentally typed svn diff in $HOME/projects/freebsd/head/sys and the output was rather interesting.

In addition to random patches that occur to me while I'm sitting on the train, I have a good chunk of syscons changes that I really should put somewhere safely.

Laptops are dangerous. Not only do they burn your lap -- and other sensitive areas -- they allow you to do work which is very easy to forget about (and then lose).

Time to sort through this mess and send out some patches.

Remko Lodder: FreeBSD Core Team

Remko — Sun, 29 Jun 2008 15:32:47 +0000

So, I recently added myself as a candidate for the FreeBSD Core Team, giving a short explaination about why I can be of help in my opinion :-). So far 27% of all people had voted, so I hope that more people will vote to get the best team possible (With or without me ofcourse).

May the best team win

Warner Losh: Adding a Serial Port to a D-LINK DIR-615

noreply@blogger.com (Warner Losh) — Sat, 28 Jun 2008 04:12:00 +0000

A while ago, I added a serial port to the D-LINK DIR-615 (HW Rev A1). Today, I'll document how I did it. First, you'll need a level converter circuit. There are many on the network. You can find a good write up on them and links from adding a serial port to a NSLU2. I always use the Parallax USB2SER that I bought years ago. It is very reliable, and I've had several provide years of good

Anders Nore: This weeks updates

andenore — Thu, 26 Jun 2008 11:00:01 +0000

This has been somewhat of a amputated week, last weekend I was away for a bicycle race and this weekend I’m going to a wedding, so it hasn’t been a lot of time for programming. Anyways I’ve added human-readable output for pkg_info’s -s option (the old is also available with the -b option). The environment variable PKG_DBCACHE_FILE is used to specify where the cache-file is located. And I’ve added features for the pkg_convert program that checks if key’s exists within the database and the printing of the data indexed in the database. The pkg_create -O option has been modified to cache information as well and installing ports from /usr/ports/ will then work if you specify the PKG_CMD in /usr/ports/Mk/bsd.port.mk to point to this pkg_create.

I’m hoping to release a test-version next week so that I can receive comments and bugs which I’m sure there is. On Thursday next week I’m going on a one week vacation to Greece so that you will have a good time to test the tools and fill my mailbox

Murray Stokely: Review: Network Administration with FreeBSD 7

Murray Stokely (noreply@blogger.com) — Tue, 24 Jun 2008 21:46:00 +0000

Packt Publishing contacted me a few months ago and offered to send me a copy of their new book, Network Administration with FreeBSD 7 by Babak Farrokhi. There is clearly a need for a modern book covering some of the newer networking features of FreeBSD, so I agreed to write a review.

The early chapters include some general FreeBSD system administration content, similar to what is provided in the FreeBSD Handbook. Chapters on disk layout, system configuration, tuning, package management, and jails precede the main networking chapters. The latter chapters include information on a number of networking technologies not well covered by the existing online documentation. For example, there are chapters or sections on GRE, OpenOSPFD, OpenBGPD, IPv6, TCP 1323, Delayed ACK, firewalls, network servers, and more.

The book is reasonably comprehensive, but there are some notable gaps. It would have been nice to see a section on SCTP or the improved wireless facilities in FreeBSD 7, for example. Also, there is a section on the Squid proxy, but the more modern Varnish project is not mentioned at all.

Unfortunately, the prose is a bit distracting as there are grammatical errors, typos, and missing articles on essentially every page. Overall this book serves as a practical guidebook for FreeBSD Network Administrators and it is a welcome contribution to the corpus of available FreeBSD books.

Ryan French: Weekly Report - 24/6/08

Ryan French — Tue, 24 Jun 2008 02:14:59 +0000

Man these weeks are flying by. I managed to get a bit of work done on MPLS, but not much, before a suprise trip to visit friends up in Auckland dragged me away for the weekend.

Accomplished Last Week:

- Began looking over the OpenBSD implementation, looking through what has been submitted on their CVS, and just generally trying to get how the code does what it needs to do, and where it does it.

- Finally got a call back with an answer from IBM about my application. They have decided not to invite me to their interview day this time, but apparently if they hold another one then they will invite me to that. I’m not holding my breath. It appears as though it is very hard to get a job in New Zealand working with operating systems or networking stuff, unless you have experience. Unfortunately for me, my grades in my first 2 years of uni where anything but flash, and I seem to be suffering for it now. It’s almost making me wish I had some PHP or web programming under my belt, which is what all the NZ jobs seem to be looking for. Oh well, onwards and upwards with the job hunt hopefully.

Plans for Next Week:

- Once I have sorted out the licencing stuff with using OpenBSDs code (I havent had anything to do with Open Source licences befire, and I dont want to step on anyones toes by doing the wrong thing) then I will be porting over the code that I need to start working on.

- Figure out where exactly the code from OpenBSD has gotten to in terms of my plans, and then start working on improving it or moving on to new features I need.

Thats all for now. If anyone knows of any jobs in NZ for someone studing networking and operating systems, let me know

Warner Losh: Old cards and the information gap

noreply@blogger.com (Warner Losh) — Mon, 23 Jun 2008 07:20:00 +0000

This time I'll explore some of the issues relating to hacking on older hardware. I'll talk about expanding the work I did for the Olicom OC-2220 to the combo OC-2232 and the problems encountered. In a previous edition, I talked about the Olicom OC-2220 card and getting it working. It works great, and in fact, I'm using it right now to make this blog entry. However, the next card I tried

Gleb Kurtsou: incompatibility and some new features

gleb — Sun, 22 Jun 2008 22:26:01 +0000

Yep. I’ve made some changes that break backward compatibility
But I’ve tried not to break anything intentionally but to do a cleanup work.

First of all most of sysctl’s responsible for layer2 filtering were replaced by per interface flags.

net.link.ether.ipfw and net.link.bridge.ipfw are replaced by l2filter interface flag.
So sysctl net.link.ether.ipfw=1 became ifconfig if1 l2tag.

net.link.bridge.ipfw_arp was renamed to net.link.bridge.pfil_layer2_arp

Introduced l2tag interface flag. It’s purpose is to add mbuf tag containing source and destination layer2 addresses to every packet passing through interface. Note that l2tag filtering against layer2 addresses is performed in layer3.

When invoked from layer2 ipfw no longer touches layer2 headers. So they following rule won’t work anymore:
ifpw allow ip from 10.1.1.1 to any src-ether 00:11:11:11:11:11 layer2

ipfw mac option was replaced by to two options: src-ether and dst-ether. ipfw still accepts mac option but translates it into src-ether and dst-ether.

Lookup tables support layer2 addresses now:
ipfw table 1 add 10.1.1.1 ether 00:11:11:11:11:11
ipfw allow ip from table(1) to any

ipfw mac-type was renamed to ether-type. Support for mac-type preserved.

Stateful filtering remains somewhat special. The problem here is that l2tag is added to a packet only in input path (when invoked from ether_demux). Such decision was intentional, mainly because it’s impossible to get tag added in output path without serious layer violations or entire pfil framework and packet handling redesign. That’s why a packet that has no l2tag attached, will pass against layer2 dynamic rule.

Dynamic rules (state created by the rule) do not check both source and destination layer2 address, but just the addresses specified by the rule created it. For example
ifpw allow ip from 10.1.1.1 to any src-ether 00:11:11:11:11:11 keep-state
will create dynamic rule that checks only source ethernet address of a packet, but not destination.

Gleb Kurtsou: post #1 (just a test)

gleb — Sun, 22 Jun 2008 14:40:55 +0000

I’m working on improving layer2 packet filtering in FreeBSD as a part of Google Summer of Code 2008.
See wiki for details.

Progress reports are to follow.

Simon L. Nielsen: The FreeBSD wiki history

simon — Sun, 22 Jun 2008 11:42:53 +0000

As I have been asked several times why we use moinmoin for the FreeBSD Developers Wiki and the answer is mainly “historical reasons” I decided to write the history up so I can just point people at it :-).

The history, as I recall it, can now be found on the WikiHistory wiki page.

Daniel Gerzo: FreeBSD Errata Notices RSS Feed

danger — Fri, 20 Jun 2008 20:15:53 +0000

Hello people,

we are working on the web based FreeBSD forums for some time now and we have added some rss feed based features to it, so that when a new item is added under News section or Security Advisories section of the www.freebsd.org page, it’s added to the approporiate forum and users can talk about the given item.
Recently, a new Errata Notice has been released (FreeBSD-EN-08:02.tcp.asc) and I have noticed that we do not have a RSS Feed for Errata notices. I have realized that it might be useful for our users to be able to subscribe to such feed so I have worked on this feature.
I have committed a few moments ago a code that brings support for Errata Notices RSS Feed to www.freebsd.org and it will be available from the following URL: http://www.freebsd.org/security/errata.xml.
So please subscribe now in order to be informed about new Errata Notices quickly and in an easy way.
This allows us to add this feed to the web based forum, so that users can disccus about errata notices as well.

More about the web based FreeBSD forums to come later.

Rui Paulo: lii(4) driver for Attansic L2 ethernet (found on Asus Eee PC 700/701)

Rui Paulo — Fri, 20 Jun 2008 01:19:44 +0000

I’ve been hacking the NetBSD lii(4) driver so that it works under FreeBSD. This driver is most notably found on the Asus line of sub-notebooks, Eee PC. So far, so good. I did not finish the porting yet, but the mechanical changes are mostly done.
The reason for this is that I bought an Eee PC 701, hence I need this driver, :-), although I haven’t touched my Eee PC yet (it’s at my parents house). But I will do the first testing this weekend.

If you have this hardware and would like to help with the effort, please email me.

The effort is being revision controlled at //depot/user/rpaulo/lii/.

Rui Paulo: tcpad status report #2

Rui Paulo — Fri, 20 Jun 2008 01:02:09 +0000

So, I found some time to continue my SoC work. tcpad is now capable of handling the most important TCP FSM transitions, like CLOSE_WAIT, FIN_WAIT_1, SYN_SENT, etc. I also implemented a basic timer facility that cleans up old connections in TIME_WAIT state. This still doesn’t honor the 2MSL required by the RFC, but it’s a start.
I also cleaned the code a little and improved the debugging macro.

Next is SEQ/ACK analysis.

Erwin Lansing: CVSROOT/modules obsoleted

erwin — Thu, 19 Jun 2008 22:12:23 +0000

After years of frustration voiced by many committers over time, Pav finally put the word out to remove CVSROOT/modules. It only adds a shorthand for checking out ports, so only the port name needs to be typed and not the full path, i.e. ports/$category/$portname. However, this can be easily achieved by a short shellscript (for other committers, see ~erwin/bin/coport on freefall for an example), and it takes quite an effort to maintain the modules file, as you can guess from its revision number of 1.20060. I would know, for some time I’ve been running a nightly script to check if all entries are done right and/or are missing and informing committers if anything is amiss. Thus, after considering the cost vs. benefit, I have emptied the file and edited addport/rmport to no longer edit it. Adding a new port was never this easy.

Oh, and it’s not every day you got to commit to a file and remove over 18.000 lines from it. Yay!