Post by Frédéric DelanoyChecking for regressions for every commit, given the amount of work
needed, isn't feasible: there are a lot of windows programs in
existence, so regressions inevitably happen... you can't test them all
for every commit, and it can be very difficult to automate, if
possible at all.
Is the regression test suite made available to devs so that they can
check their code changes before submitting them? If not, maybe it should
be.
There are regressions tests for most of the DLLs in Wine, and code
doesn't get committed if tests fail.
Now granted they appeared fairly late in the history of Wine
development (2003), after the "alpha period" (wine-yyyy-mm-dd
releases) [Wine project began in 1993]
There's also http://newtestbot.winehq.org/ so programmers can test
their tests on a variety of Windows OSes, and
http://test.winehq.org/data/
Also sometimes the same code works differently between different
Windows releases (ex: 9x, 2000, vista), but Wine obviously can't
implement all behaviours (though you can set the Windows version Wine
mimics through winecfg). Now remember Wine purpose is to run Windows
applications, not mimic a particular Windows version.
Post by Frédéric DelanoyThere are a lot of unit tests in the code though, which are run for
every commit. And FYI non-compiling code isn't committed...
And only the project leader has commit rights.
I was assuming that the regression test suite would mainly consist of
full coverage for each API implemented by Wine, i.e. would cover corner
cases and errors as well as calls that should work, together with some
tests for common or known troublesome API call sequences, rather than
running actual applications.
That's how it's done. Cf. http://wiki.winehq.org/UnitTestSuites.
Though there are some limited support for automated Windows apps
tests, AppInstall (cf. previous link)
Post by Frédéric DelanoyKicking developers out for every regression... If one would do what you
suggest, I guess you would be the only Wine developer remaining?!?
Good way to kill a project...
Well, I would expect devs to have done pretty solid testing on anything
they submit and to include any new tests aloing with the code so the
tests can be included in the regression test suite. If somebody doesn't
do that and persists in not doing it despite getting suggestions and
advice on how to improve his testing, would you really want them on the
project?
If someone sends patchs which fails tests repeatedly, be sure their
patches would soon be redirected to /dev/null
The maintainer, Alexandre Julliard, is actually enforcing strict rules
for code inclusion, and even seasoned developers get their patches
rejected from time to time (passing tests is a necessary, not
sufficient condition), especially when a stable version is
approaching.
Note though that tests can only show the presence of bugs, not their absence.
You're producing code that, like Fedora or any of the 'unstable' Linux
releases, feeds into the commercial Crossover product, so at least some
of the standards and practises used in commercial code development
should apply.
FYI, many of the core Wine developers are actually Codeweavers employees.
Post by Frédéric DelanoyIf you want to avoid regressions at all costs, stay with stable branch
(1.6.x). Use development versions at your own risk.
Noted. My comments apply to past experience with the versions that have
been packaged and released as part of RedHat Fedora upgrades which would
therefore seem to be the stable branch since 1.6.1 is the most recent
version to have been released as part of Fedora.
Not necessarily. Can't speak about Fedora, but distribution packagers
sometimes include stuff not present in vanilla wine (e.g.
winepulseaudio)/
To get latest development Wine packages, one mostly has to rely on
stuff like PPAs, except maybe for rolling releases like Gentoo.
Post by Frédéric DelanoyPost by Martin Gregorie(2) publishing a version that fails regression or has incomplete changes
testing is a really good way to destroy its reputation. Gnome 3 anybody?
IIRC the issues with Gnome 3 was released as a stable version with a
lot of bugs. Wine development releases ARE NOT stable versions.
Same case. I was getting Gnome 3 as part of Fedora upgrades. As you say,
it was buggy, as were the final releases of Gnome 2 that preceded it.
However, there I had a solution: move to XFCE, which I did.
Me too actually ;) Although I got to XFCE when Ubuntu broke stuff
with their Unity crao.
Post by Frédéric DelanoyNobody forces you to upgrade your Wine version. You can keep an old
Wine version for some apps, and even distinct versions on the same
system (as a developer, this shouldn't be difficult for you).
Not entirely true. AFAIK there's no way to tell yum to avoid upgrading
Wine or any other package in the main Fedora repositories.
You can always compile your own wine versions in their respective
directories, and e.g. start wine from there.
It's of course not that easy for plain users.
Post by Frédéric DelanoyMoreover, if you find a regression, please report it on
http://bugs.winehq.org/. Not everybody runs the same applications as
you do, again checking every app in existence is just unrealistic.
Fair comment.
A problem we have also is to get users to perform regression tests
(http://wiki.winehq.org/RegressionTesting). Some also tend to do stuff
like use unsupported third-parties, like PlayOnLinux, copy native
dlls, ..., which sometimes makes troubleshooting/but triaging a PITA.
Post by Frédéric DelanoyThey do exist, and are very much enforced. Look e.g. on the wiki for more
information.
I'm glad to hear it.
Post by Frédéric DelanoyFeel free to suggest improvements on Wine development
standards/procedures on the wine-devel mailing list, or on IRC
(http://wiki.winehq.org/IRC).
Noted, though my Wine use is fairly limited: I chiefly use it to run
SPINE, which pulls down and displays NOTAMs. SPINE is about the only
Wine app I run on a weekly or daily basis.
You might want to check the AppDB (http://appdb.winehq.org/) to see if
it contains an entry for your specific app, and add one if necessary.
Of course, good bug reports are also appreciated
(http://bugs.winehq.org/ ; http://wiki.winehq.org/Bugs)
In addition I use it every
few months to check out new versions of glider navigation software
(XCSoar, LK8000 and SeeYou). These are normally run on WinCE 5/6 based
PNAs, but all have Windows versions for training, evaluation and
checking new versions of the data files they depend on (maps, restricted
airspace, turnpoints, etc.
As this is fairly minority stuff it probably doesn't fit will with
WineDB.
A small problem can hide a bigger issue. Don't hesitate to report bugs
for that. If a bug isn't known, it's difficult to fix.
Best regards,
Martin
Regards,
Fr?d?ric