Björn Tillenius

An LLM wrote 100% of my code

Sun, 15 Mar 2026 09:00:00 +0000

As many of you, I often see the claim that an LLM, or rather an LLM agent framework (like Claude Code or GitHub Copilot), wrote 100% of someone’s code.

I’ve always been curious whether this could be true or not. I started playing around with using LLMs in my daily work as a software engineer at the end of last summer. In the beginning it was truly a struggle. I used an LLM to write code, but I also had to spend a lot of time reviewing and refactoring the code it generated. At that time, it also made simple mistakes, like redefining variables or functions, generating code that wasn’t correct, and so on.

However, even though I didn’t feel that I was more productive using an LLM than writing the code myself, I continued trying to make use of LLMs in my work, trying different models, different frameworks, learning how to prompt, and so on. I learned a lot, but I’ve also seen a great improvement in the agent frameworks that are out there. For example, they can now connect to an LSP server and run unit tests automatically, so they can verify and fix their code, before they give it to me to review.

I’m now at a point where for a couple of greenfield projects, I’ve used an LLM and agent framework to write 100% of the code. I finally feel that I’m productive with an LLM agent than without.

I now also know that the claim that an LLM wrote 100% of the code can in fact be true.

100% writing code != 100% engineering

It should be pointed out that usually the claim is that the LLM wrote 100% of the code. Writing code is only one part of the engineering process. When using LLMs for software engineering, it’s important to understand what an LLM is great at doing, and what it’s not great at doing.

For example, an LLM is great at generating code quickly. It can write code much faster than any human out there. However, it’s not great at keeping a large codebase maintainable. At least not without guidance from a human software engineer.

An LLM is great when the instructions are clear, and there are already well-known solutions to the problems. They are great at inferring complicated patterns, but they often fail to understand subtle nuances. A great example of that is the “Car wash” problem, which proves to be challenging to understand for an LLM. It basically goes like this:

You: I need to wash my car. The car wash is 300 meters away. Should I drive or walk there?
LLM: Walk

The LLM focuses on the most common problem, driving or walking, but fails to understand the nuance that the car needs to get to the car wash somehow. Even frontier models like Claude Opus 4.6 get this wrong. I tested it today, and it got it slightly better, but still failed:

That's a fun little dilemma! For 300 meters, I'd say walk — that's only about a 3–4 minute stroll. Driving such a short distance barely warms up the engine and uses fuel for no real benefit.

That said, if you need to drive the car through an automatic car wash (the kind where you stay in the car), then obviously you'd need to drive it there. But if it's a self-service or hand wash place, walking over is the way to go — maybe even enjoy the fresh air on the way.

Is this someone that you would trust to make important decisions about your architecture and codebase?

Agent feedback loop

I’m not going to go into details here exactly what I’m doing. I might write another blog post about that in the future. The details are important, but I also don’t think there is a single best way of doing things. I think people need to try out different approaches and see what works best for them. I did enjoy reading what Mitchell Hashimoto wrote, and I feel that my story is quite similar.

However, something that I do think is important is that you have a feedback loop with your LLM agents. It basically goes like this:

Now, let’s go through each step in more detail.

Step 1: Plan

This is where you, as a software engineer, make the architectural decisions and create a plan for the LLM agent to follow. The better plan you make, the better the LLM agent will perform.

This step doesn’t have to be completely manual. You can make use of an LLM agent to help you create the plan. The important thing is that the plan is something that you take accountability for. You are the one that is creating the plan, even if you get help from an LLM agent.

Step 2: Delegate

When you have a plan, you can give it to an LLM agent to execute. Again, I’m not going to go into details here, because there are many ways of doing this. For example, most likely you will divide up the plan into smaller tasks, and give the LLM agent one task at a time.

For each task, you can also have a feedback loop, so that you can give course corrections early if you see that things don’t go the way you expected.

Step 3: Review

This is probably the most important step. It’s now that you verify that the LLM agent wrote the code in the way that you expected. Remember that you are the software engineer. You understand what the code needs to look like in order to keep the codebase maintainable and scalable.

This is where I currently spend the most amount of time. I go through the code carefully, and make sure that it matches what I would have written myself. Sometimes it’s the LLM agent that did something unexpected. Sometimes it’s the plan that is lacking in details. But sometimes it could be that I only now realize that the code should be structured differently.

It’s also not about code structure and architecture. It includes running the code to make sure it works, do a security review of the code, checking for subtle bugs, and so on.

Remember, you can still use an LLM agent at this step as well. For me, it helps me produce better quality code, since doing large refactorings before I submit a PR is much easier than it used to be.

The goal of the feedback loop is to reduce the amount of time you spend here.

Step 4: Refine

This is how you reduce the amount of time you spend in Step 3. You need to look at what you needed to do in Step 3 and try to improve things so that the LLM agent can do a better job in the future.

Again, I’m not going into details here, since there are many ways of doing this, and it might depend on whether you use Claude Code, Codex, GitHub Copilot, or something else.

But you should be aware of how your LLM agent gets the context it needs in order to perform its task. I will give some examples here.

Prompt

The prompt is the most obvious one, since it’s usually how you tell the LLM agent what to do. You should think about how you write the prompt, but since this is something that you need to do for every feedback loop, it’s one of the least scalable solutions.

PLAN.md

For larger tasks, it’s often helpful to create a document that contains the plan, with the general outline what needs to be done. But it can also contain example code, breakdowns of the task into subtasks, links to relevant documentation, and so on.

Take some time to reflect on whether your PLAN.md could have been better.

AGENTS.md

A lot of LLM agents automatically read AGENTS.md if it exists for your project. This is a great place to put project-specific instructions. For example, if you see that the LLM agent added a lot of useless comments all over the place, AGENTS.md is a great place to add a note that comments should be added only to explain non-obvious code.

As long as the rules make sense on a project level, this is a good place to put them.

Skills

Modifying AGENTS.md is not always the answer, since you need approval from other team members. For rules that don’t have a team-wide consensus, you can create personal prompt files. Something that automatically gets included in every prompt you write.

There are again many ways of doing this, agent skills is one example. Some LLM agents support it, some don’t.

The idea is that you can create a library that guides the LLM agent for specific tasks.

In the case of agent skills, they can either live locally on your filesystem only, or be included in the project repository, as a complement to AGENTS.md.

Documentation

Another way of guiding the LLM agent is to make sure that your code and architecture are well-documented. LLM agents usually can read documentation when they need to get more context, and you can also reference the specific documentation in your plan.

This is something that helps both LLM agents and humans to understand your codebase.

Vibecoding

When talking about an LLM writing 100% of the code, it’s hard to not think about vibecoding. I see a lot of people talking about it, and I also feel that some people don’t really understand what it means.

What I talk about in this post is not vibecoding. Vibecoding is a term coined by Andrej Karpathy. Basically, they define it as using LLMs to write code, where you trust the LLM to produce the correct code. You give the responsibility of maintaining the codebase to an LLM agent.

If you’ve read this far, it should be clear that what I’ve been talking about in this post is not vibecoding. I’m not yet convinced that an LLM agent can maintain a large codebase in the long term, at least not currently. The LLM agents and models do improve rapidly, but I still feel that they have a long way to go in this area.

Vibe coding vs agentic coding

Agentic coding is basically when you move away from using the LLM models through a basic chat interface. Instead you use something like Claude Code that has agents that can look at your codebase, talk to LSP language servers, run unit tests, and even interact with other agents.

Agent engineering has improved the quality of the state of code, but it’s not really related to vibecoding. Even if you use agent engineering, you can still choose whether you do vibecoding, where you trust the LLM agents fully. Or you can choose to not do vibecoding, and instead take ownership of the codebase yourself.

Is vibecoding always bad?

Of course not! Let’s say that you’re writing a one-off script, doing a functional prototype to show off some workflow, or small projects that you don’t anticipate will be maintained over time.

In that case, vibecoding can be a great accelerator, since you will save a lot of time compared to writing it manually, or carefully reviewing the code.

Using smbus with Python 3 on a Raspberry Pi

Sun, 06 Sep 2015 15:50:00 +0000

A while ago I got an Orange Matchbox, which is a Raspberry Pi running Snappy Ubuntu Core together with an Ubuntu branded case and a PiGlow. The Snappy image for Raspberry Pi recently got support for I2C, so I tried to drive the PiGlow using Python 3. It turned out it wasn’t trivial, since python-smbus doesn’t support Python 3 out of the box. Further more, you can’t install .debs in Snappy, so installing python-smbus for Python 2 doesn’t work either. And it’s a C extension, so you can’t just copy over the sources.

I’ve seen some posts about how you can patch the python-smbus source code and recompile it for Python 3. That’s a lot of work when you just want to try and play around a bit. I instead took a stab at reimplementing python-smbus in pure Python. You can find the result here:

https://github.com/bjornt/pysmbus

It only implements two methods so far, read_byte_data and write_byte_data, but that goes a long way. I also patched the PyGlow package to support Python3 and not to require RPi.GPIO, which also is a C extension I don’t want to recompile:

https://github.com/bjornt/PyGlow/

With these two packages, we can now drive the PiGlow using Python 3 in Snappy Ubuntu Core. First we assemble the code on our desktop machine and copy it to the Raspberry Pi:

desktop:~> git clone https://github.com/bjornt/PyGlow/
desktop:~> cd PyGlow
desktop:~/PyGlow> wget https://github.com/bjornt/pysmbus/raw/master/smbus.py
desktop:~/PyGlow> scp -r ../PyGlow ubuntu@pi2:

Then, on the Raspberry Pi, we run the bin_block.py example. Note that we have to use sudo, so that we get permission to access the I2C bus:

desktop:~/PyGlow> ssh ubuntu@pi2
(RaspberryPi2)ubuntu@localhost:~$ cd PyGlow
(RaspberryPi2)ubuntu@localhost:~/PyGlow$
(RaspberryPi2)ubuntu@localhost:~/PyGlow$ sudo PYTHONPATH=. python3 examples/bin_clock.py
 Hour | Minute | Second
01111 | 011001 | 111010 (15:25:58)

If we look at our Raspberry Pi, we should see the PiGlow’s lights working:

VLAN tagging with Linksys E8350

Sun, 31 May 2015 14:00:00 +0000

I bought a Linksys E8350 router to replace the router my ISP gave me as well as an old Linksys WRT610N router. I chose the E8350, since from the setup screenshots it looked like it could do VLAN tagging, which I need. I was disappointed at first, since the UI doesn’t allow you to do VLAN tagging for the LAN ports. But it turned out that with a little bit of hacking, it’s possible to do it.

Doing VLAN tagging with 5 ports is quite limiting, but just about enough for my use case. I get a, somewhat odd, trunk from my ISP which has a VLAN for IPTV traffic, but regular Internet packets aren’t tagged. I have the router in my bedroom and want to access both the LAN and IPTV networks in my living room. But I have only a single cable connecting the rooms, so I use VLANs to be able to connect both networks with one cable.

The first attempt to make port 4 the trunk for the IPTV and LAN VLANs failed:

It’s odd, because the UI looks like it would allow creating a trunk on the LAN ports. A closer look at the Javascript that does the validation revealed this:

    for(i=1;i<VLAN_ENTRY;i++)
    {
        obj = getEle(ele_id_name[ele]+i);
        if(obj.value != port_option[2])
        {
            if(flag == true){
                //[share.js]errmsg.err104="A VLAN port can be untagged to only one VLAN ID.";
                alert(errmsg.err104);
                obj.focus();
                return false;
            }
            flag = true;
        }
    }
    return true;

The commented out error message makes sense. A port can’t be untagged in different VLANs. But the actual code checks also that a port can’t be tagged in different VLANs, and that doesn’t make much sense. It could be a bug, but considering that the actual error message that was displayed doesn’t mention ‘untagged’, it could be that they took an existing VLAN UI from some other product, and just crippled it. The E8350 is a consumer model after all.

Now, what would happen if the validation above would be disabled? The Javascript is in the HTML file, and Chromium Developer Tool doesn’t allow such Javascript to be edited. Instead, let’s put a breakpoint whenever flag is set to true and use the Javascript console to set it back to false.

Look at that, it worked!

Enabling middleclick scrolling in Ubuntu for Lenovo clickpads

Tue, 19 Aug 2014 08:30:00 +0000

The short version is that if you want to enable middleclick scrolling for Lenovo clickpads in Ubuntu, do this in a terminal:

sudo add-apt-repository ppa:bjornt/evdev
sudo apt-get update
sudo apt-get dist-upgrade

The commands above should upgrade the xserver-xorg-input-evdev package, as well as remove the xserver-xorg-input-synaptics and xserver-xorg-input-all packages.

Next you need to create a file at /usr/share/X11/xorg.conf.d/90-clickpad.conf with the following contents:

Section "InputClass"
    Identifier "Clickpad"
    MatchIsTouchpad "on"
    MatchDevicePath "/dev/input/event*"
    Driver "evdev"
    # Synaptics options come here.
    Option "Clickpad" "true"
    option "EmulatedMidButtonTime" "0"
    Option "SoftButtonAreas" "65% 0 0 40% 42% 65% 0 40%"
    Option "AreaBottomEdge" "0%"
EndSection

Section "InputClass"
    Identifier   "TrackPoint"
    MatchProduct "TrackPoint"
    MatchDriver  "evdev"
    Option       "EmulateWheel"       "1"
    Option       "EmulateWheelButton" "2"
    Option       "XAxisMapping"       "6 7"
EndSection

The interesting options are SoftButtonAreas and AreaBottomEdge. SoftButtonAreas specifies where the buttons should be. If you want the buttons at the top, itt should generally be in the form "R 0 0 H L R 0 H", where R is where the border between the middle and right buttons, H is the height of the buttons, and L is the border between the middle and left buttons.

The AreaBottomEdge turns off the touchpad, expect for clicking. If you want to keep using the touchpad, you can instead specify AreaTopEdge, with the same value you use for H. That would enable the touchpad below the buttons.

Unfortunately, you can’t specify where the left button should be, instead if occupies everything that isn’t the middle or right button. This is a bit annoying, since at least I tend to touch the touchpad with my palm when reaching for the middle button, which will result in a left click being registered instead of a middle click.

I created this package because Ubuntu doesn’t quite support the clickpads that come in the newer Lenovo laptops. Ubuntu does support clickpads, and with the SoftButtonAreas config settings it’s possible to have three soft buttons on the clickpad where the real buttons used to be. However, what’s not supported out of the box is middleclick scrolling, where you hold the middle button and scroll with the trackpoint.

The main problem is that the clickpad is driven by synaptics and the trackpoint by evdev, and they can’t communicate to generate the scroll events. Bae Taegil patched the evdev driver to basically include the synaptics driver. I’ve taken that patch and generated a package for Ubuntu 14.04. I’ve only added a package for Trusty, but I could add packages for other releases if needed. I will most likely add one for Utopic, when it becomes more stable.

Using web.go on Google App Engine

Mon, 07 Nov 2011 13:50:46 +0000

In short, if you want to use web.go in Google App Engine’s Go runtime environment, check out my google-app-engine branch of web.go. Using that one you can start using web.go like this:

package webgoexample

import (
    "http"
    "log"
    "os"
    "web"
)

var server *web.Server

func init() {
    server = &web.Server{
        Config: web.Config,
        Logger: log.New(os.Stdout, "", log.Ldate|log.Ltime)}
    server.Get("/", func(ctx *web.Context) {
        ctx.Write([]uint8("Hello from web.go!"))
    })

    // Send all requests to web.go.
    http.HandleFunc("/", handler)
}

func handler(writer http.ResponseWriter, request *http.Request) {
    server.ServeHTTP(writer, request)
}

BTW, if you simply put the web.go branch in your root dir, you have to remove the examples/ directory, otherwise App Engine won’t be able to compile your project.

The branch in question has quite minimal changes to web.go, but I haven’t proposed to merge it to trunk yet, since it removes some functionality. First of all I changed it not to set up /debug/ paths, so that I could remove the use of http/pprof, since it’s not available on App Engine. After that I had to also remove the use of net.ResolveTCPAddr, which is also not available on App Engine. I basically replaced it with net.SplitHostPort, which I suspect is good enough. It doesn’t resolve host names and port names, but I’d be surprised if hr.RemoteAddr wouldn’t be an IP address and a port number.

formataddr() and unicode

Fri, 11 Feb 2011 14:44:07 +0000

I often see code like this:

message["To"] = formataddr((name, email))

This looks like it should work, especially since the docstring of formataddr() says that it will return a string value suitable for a To or Cc header. However, while it works most of the time, it fails if name is a unicode string containing non-ascii characters. It may look ok if you look simply read message["To"], but as soon as you convert the message or header to a byte string, you will see the problem.

>>> from email.Message import Message
>>> from email.Utils import formataddr
>>> msg = Message()
>>> msg["To"] = formataddr((u"Björn", "bjorn@tillenius.me"))
>>> msg["To"]
u'Bj\xf6rn <bjorn@tillenius.me>'
>>> msg.as_string()
'To: =?utf-8?b?QmrDtnJuIDxiam9ybkB0aWxsZW5pdXMubWU+?=\n\n'

Most code that will use the To address in the example will fail, since there’s no visible e-mail address in there. The header should look like this, i.e. only the name itself should be encoded:

To: =?utf-8?b?QmrDtnJu?= <bjorn@tillenius.me>

I wish Python would handle this better. I usually end up writing a helper function like this for projects I work on:

def format_address(name, email):
    email = str(email)
    if not name:
        return email
    name = str(Header(name))
    return formataddr((name, email))

Problems with drive-by fixes

Mon, 03 May 2010 12:10:47 +0000

To start with, I think drive-by fixes are great. If you see that something is wrong when fixing something else, it can be a good idea to fix it right away, since otherwise you probably won’t do it.

However, even when doing drive-by fixes, I still think that each landed branch should focus on one thing only. As soon as you start to group unrelated things together, you make more work for others. It might be easier for you, but think about all the people that are going to look at your changes. Please, don’t be lazy! It doesn’t take much work to extract the drive-by fix into a separate branch, and most importantly to land it separately. If you do find that it’s too time-consuming to do this, let’s talk, and see what is taking time. There should be something we can do to make it easier.

There’s no such thing as a risk-free drive-by fix. There’s always the potential of something going wrong (even if the application is well-tested). When something goes wrong, someone needs to go back and look at what was done. Now, if you land the drive-by fix together with unrelated (or even related) changes, you basically hide it. By reducing your workload slightly, you create much more work for someone else.

For example, on Friday I saw that we had some problems with scripts in Launchpad. They were trying to write to a mail directory, to which they didn’t have access. That was odd, since scripts have always talked to the SMTP servers directly, and didn’t use the queued mailer that needed write access to that directory. Looking through the recent commit logs didn’t reveal anything. Luckily enough, William Grant pointed out that r9205 of db-devel contained a change to sendmail.py, which probably was the cause of the problems. This turned out to be correct, but it was still pretty much impossible to see why that change was made. I decided that the best thing to do was to revert the change, but I wasn’t sure exactly what to revert. That diff of that revision is more than 4000 lines, and more than 70 files were changed. So how can I know which other files were change to accommodate the change in sendmail.py. I tried looking at the commit logs, but that didn’t reveal much. The only thing I could do was to revert the change in sendmail.py and send it off to ec2, waiting three hours to see if anything broke. So, I plead again, if you do drive-by fixes (and you should), please spend a few minutes extra, to extract the fix into a separate branch, and land it separately! Is there maybe anything we can do to make this easier to do?

RFC: Expose features to users when they are done, not once a month

Thu, 04 Mar 2010 18:56:41 +0000

I’ve been working on a new release/merge workflow for Launchpad. I’ve written it from the developers’ point of view, but I’d love some comments from users of launchpad.net, so let me try to explain how you, as users, would be affected by this change.

The proposal is that we would decouple our feature development cycles from our release cycles. We would more or less get rid of our releases, and push features to our users when they are ready to be used. Every feature would first go to edge.launchpad.net, and when it’s considered good enough, it will get pushed to launchpad.net for everyone to use. Bug fixes would also go to edge.launchpad.net first, and pushed to launchpad.net when they are confirmed to work. Sadly, Launchpad will still go down once a month for updating DB and other regular maintenance, just like before. The amount (and frequency) of downtime would stay the same as before.

There are users who are in our beta team and use edge.launchpad.net all the time, and those who want a more stable Launchpad, and use launchpad.net.

Users of `launchpad.net`

Those who aren’t in the beta team would get bug fixes sooner than with the current workflow. Instead of having to wait for the end of the release cycle, they will get it as soon as the fix has been confirmed to work on edge.launchpad.net. The same is true for features, kind of. These users would have to wait a bit longer than today, since today we push even unfinished features to launchpad.net users at the end of the release cycle. With the new workflow, these users would have to wait for the feature to be considered complete, but in return these user should get a better experience when seeing the feature for the first time.

One potential source of problem is that even though fixes and features get tested on edge.launchpad.net, before going to launchpad.net, with each update there is the potential of some other issue being introduced. For example, fixing a layout bug on one page, might make another page look different. With the current workflow this happens only once a month, instead of a few times every month with the new workflow. That said, even today we update launchpad.net multiple times every month, to fix more serious issues.

Users of `edge.launchpad.net`

If you are in the beta team, and use edge.launchpad.net on a regular basis, it won’t be that different from how it works today. Just like today, you would be exposed to features that are under development. What would change is that we will try to do a better job at telling you which features that are on edge.launchpad.net only. This way you will have a better chance at actually helping us test and use the features, and tell us about any problems, so that we can fix it right away. This should make you more aware of new features that are being added to Launchpad, and provide a better opportunity for you to make it better.

One potential source of problem here is that developers will know that their work won’t end up on launchpad.net, before they say it’s ready, so they push more rough features to edge.launchpad.net. Thus it could be a more rockier ride than today. But of course, our developers care a lot about their users, so they won’t land their work, unless it’s really good! :-)

Conclusion

My hope is that this will provide a better and stable experience for users of launchpad.net, and provide users of edge.launchpad.net a better opportunity to help us making new features rock! But I’m interested to hear what you, the actual users, think about this.

Technical Architect Roadmap

Wed, 24 Feb 2010 12:12:23 +0000

I made the transition from the Bugs team lead to the Launchpad Technical Architect quite a while ago. While the first time was spent mainly on satisfying my coding desires, it’s time to define what I’m going to spend my time as technical architect! My road map that shows the high level things that I’ll be focusing on is available here:

https://dev.launchpad.net/TechnicalArchitect

I’ll also be writing blog posts (and sending mails to the launchpad-dev mailing list of course) regularly to keep people updated with my progress and findings. My blog is at http://tillenius.me/ and I tag all posts related to Launchpad with launchpad.

I’m currently working on decoupling our feature development cycles with our release cycles, which I do mainly, because I think it’s important, not because it’s part of the technical architect’s responsibility. But in parallel with that my next task is to set up a team that can help me doing a good job. I’ll expand more about the team in another post, but in short it will consist of members from each sub-team in Launchpad. It will act as a forum to discuss what needs my attention, and they will also help me figuring out solutions to problems, and help me implement the solutions.

One of the first major tasks will be to come up with a testing strategy. Currently when we write tests we don’t think that much about it. Everyone have their preferences, and we have a wide variety of testing styles, making it hard to find which code paths are exercised by which tests, and how good test coverage we have. This leads to us sometimes having bad test coverage, and some times having too much test coverage, i.e. we have redundant tests that make the test suite slower. Coming up with guidelines on how to write tests, which kind of tests to write, where to place them, etc., is the first step. But we also need to figure out how to make our test suite faster, what kind of documentation to provide, and so on.

In addition to the tasks on the roadmap, I also have a number of things I do on a regular basis. This includes reviewing database patches for API consistency, help teams design features from a technical point of view, keep my eyes open for areas in the code that need refactoring and clean-up.

Windmill - so far, so good

Wed, 10 Feb 2010 12:51:17 +0000

We’ve used Windmill in our Launchpad buildbots for a while now, and it’s actually worked out quite well. I was afraid that we would have a lot of fallout, since in the beginning Windmill was fragile and caused a lot of intermittent test failure. However, so far I’d said that we’ve had very little problems. There was one intermittent failure, but it was known from the beginning that it would fail eventually. Apart from that we’ve had only one major issue, and that’s that something is using 100% CPU when our combined Javascript file is bigger than 512 000 bytes. This stopped people from landing Javascript additions for a while, and we still haven’t resolved this issue, apart from making the file smaller.

There are some things that would be nice to improve with regards to Windmill. The most important thing is to make sure that launchpad.js can be bigger than 512 000 bytes:

https://launchpad.net/bugs/519744

It would also be nice to make the test output nicer. At the moment Windmill logs quite a lot to stderr, making it look like the test failed, even though it didn’t. We don’t want Windmill to log anything really, unless it’s a critical failure:

https://launchpad.net/bugs/494519

I was going to say that we also have some problems related to logging in (because we look at a possibly stale page to decide whether the user is logged in), but it seems like Salgado already fixed it!

https://launchpad.net/bugs/515494

It would also be nice to investigate whether the problem with asserting a node directly after waiting for it sometimes fails. We had problems like that before; code was waiting for an element, and when using assertNode directly after the wait, the node still didn’t exist. I haven’t seen any test fail like that lately, so it might have been fixed somehow:

https://launchpad.net/bugs/487666

There are some other things I could think of that would be nice to have. I haven’t found any bugs filed for them, but I’ll list them here.

Don’t run the whole test suite under xvfb-run. It’d be better to start xvfb only for the Windmill tests.
Use xvfb by default for Windmill tests. When running the Windmill tests it’s quite annoying to have Firefox pop up now and then. It’d be better to run them headless by default.
Switches for making debugging easier. Currently we shut down Firefox after running the Windmill tests. It should be possible to have Firefox remain running after the test has finished running, so that you can manually poke around if you want to. If we use xvfb by default, we also need a switch for not using it.