Bug 20313 (Hazel)

Is anybody there?

Howdy, looking at the mailing list link you sent, there certainly didn't seem like agreement. There was more a suggestion to use and existing library. What will make your library different and at what point is your library currently? Ie do you have any code created? If so can you provide links and documentation?

We get lots of requests at FD.o for projects so we have to make sure requests are viable.

ping

Sorry the delay, i have been out most of the summer.

I have recently published the version 0.2 of this library, it's called libsysactivity and it's placed here: https://sourceforge.net/projects/libsysactivity/

I just think it should be in fd.o because it's usable by all the desktops.

You can also check the documentation here: http://sourceforge.net/projects/libsysactivity/files/Documentation/

Please, let me know if you have any doubt.

vincent, any comments?

Let me ask Benoît Dejean, the libgtop maintainer (libgtop is the library that we use for this in GNOME). I'm not sure he has an account on this bugzilla, so I'll send him a mail.

Btw, Carlos, any chance you already contacted Benoit?

I've just pointed both Benoit and ksysguardd maintainer (John Tapsell) to this bug so i guess they will talk here soon.

Hello everyone,

libgtop is not perfect, but it already provides good support for many platforms. I never found the time to revive the network code that would fully make it able to replace the ksysguardd backend.

I know the ksysguardd backend, i know it's problem, everything is mixed while libgtop has everything nicely split. Rewriting everything and providing a working port for linux/*bsd/solaris is a __HUGE__ work.

Also, it is very important to design an API that's not too linuxish because it would make the port harder. But on the other hand, splitting the API often provides lower performance. For example, you can retrieve memory usage and swap at once on many systems, but if you split the API like libgtop does, you're going to do 2x the work.

There's a lot of work to be done around libgtop, maybe it's to soon to kill it.

libsysactivity is not complete but we can hold things until it's ready and plan on it. In fact, i have to say that linux/FreeBSD/Sun OS support is already done and i'm working hard on Open/NetBSD and windows support, so the huge work is almost done!

The main difference between libsysactivity and libgtop/ksysguardd is that it was planned with a desktop-independent idea in mind. The API is there, you can see what i mean ;)

Benoît Dejean, would you see libsysactivity as a worthy replacement in the long term? - I'm trying to gauge if libsysactiviy should become a long term supported fd.o project.

To celebrate the first birthday of this bug I want to announce the release 0.5.0 with full Free/Open/NetBSD support so the remaining OS are: MacOS X and windows.

Also, take a look at the improvements on the API for retrieving processes stats.

Hi, version 0.5.4 has been released.

_A lot_ of changes have been commited to libsysactivity since 0.5.0 but the bigger ones are:
  * The code is now from 10% to 60% faster.
  * Unit and stress tests: automated tests are used to guarantee the quality of the code.
  * The documentation is much more detailed: SMP, matrix of capabilities per OS...

Also there are packages for debian/kFreeBSD, ubuntu, debian (among others).

Any comment about the future of libsysactivity on fd.o?

Hello everyone,

About libsysactivity, that's basically a nano-libgtop library. Here are some few thoughts. I'm not advocating for libgtop, i'm comparing with something i know.

What i like:
- the project is so small, the implementation is then simple.
- the API is simple and just like libgtop.
- it's thread-safe (although i don't really get the point of keeping open files, but why not. But that sticks with the kstat on solaris.)
- cmake
- doxygen

What i don't like:
- does not solve any libgtop issues, especially it is also based on C-structs which requires to break the ABI whenever you want to add a new struct member. But C-structs are fast.
- some parts are not resilient at all (eg. the linux swap seek(fd, 37, ...) on /proc/swaps). Linux /proc changes a lot so the library has to be resilient to handle multiple version of the kernel.
- my experience told me that stdio FILE and sscanf are incredibly slow.
- i'm not sure everyone has a C99 compiler for <stdint.h>. The GNOME boy in me sticks to Glib datatypes.
- nothing for remote monitoring.

The API is also very small (not much info) so i don't think i could rewrite gnome-system-monitor on it.
Nothing about SMP, memory mapping, etc, etc.

(In reply to comment #15)

Well. All of this was true many months ago. MANY things have changed since then.

> Hello everyone,
> 
> About libsysactivity, that's basically a nano-libgtop library. Here are some
> few thoughts. I'm not advocating for libgtop, i'm comparing with something i
> know.
> 
> What i like:
> - the project is so small, the implementation is then simple.
> - the API is simple and just like libgtop.
> - it's thread-safe (although i don't really get the point of keeping open
> files, but why not. But that sticks with the kstat on solaris.)
> - cmake
> - doxygen

You can add unit tests here ;)

> 
> What i don't like:
> - does not solve any libgtop issues, especially it is also based on C-structs
> which requires to break the ABI whenever you want to add a new struct member.
> But C-structs are fast.
> - some parts are not resilient at all (eg. the linux swap seek(fd, 37, ...) on
> /proc/swaps). Linux /proc changes a lot so the library has to be resilient to
> handle multiple version of the kernel.

Now it detects the version of the running kernel and it's not seeking to a fixed position anymore.

> - my experience told me that stdio FILE and sscanf are incredibly slow.

It is! sscanf and many other slow code was removed from the code and improvements (from 10% to 40%) were measured with stress tests.

> - i'm not sure everyone has a C99 compiler for <stdint.h>. The GNOME boy in me
> sticks to Glib datatypes.

But i have to be desktop agnostic. Anyway, the only compiler that i found that doesn't have <stdint.h> is Visual Studio prior to 2010... so it's not a problem.

> - nothing for remote monitoring.

This kind of code is out of the scope of the project, but libsysactivity is intended to be a versatile library so you can use it in such projects.

> 
> The API is also very small (not much info) so i don't think i could rewrite
> gnome-system-monitor on it.
> Nothing about SMP, memory mapping, etc, etc.

Well, the project supports SMP when it's supported on the underlying OS. All about this was added to the documentations some months ago.

I actually wrote it months ago but forgot to post.
The main issue is still C-struct. I don't have any good solution, but the least would be to add some reserved fields. Linux keeps adding more and more information in /proc. For example, /proc/memory has 41 lines on my laptop and your memory struct has only a dozen of members. If someone comes and asks you to also report info at line #33, you're screwed, you need to break the ABI :/

If somebody asks for a new field i will check whenever the field has a meaning that worths to be included in the library or, by contrast, it's too specific.

I don't want to brake the ABI but if i don't find any solution for the C-structs it's something that would happen. But, what is worse? breaking the ABI (just sometimes) or the only ugly solution that i can see for the C-structs?

If i can't find any really good solution for the C-structs, why not break the ABI? Maybe it's the least tradeoff.

Well that's just my experience. You first break the ABI, you think it's no big deal. Then you see distro lagging behind, etc. Then you break it again, for good you think. But 3 days after your release, you realize that you should add something more.

I don't have a solution, i'm just warning you.
At least, you should have some reserved members, just in case.
It costs nothing.

For example about memory, if tomorrow linux provides a fast way to get the real memory usage (that you can actually compute from /proc/*/smaps) and writes it to /proc/meminfo. Or a new member about memory used by virtual machines. Etc.

Another example: "please add swap pagein/pageout to sa_memory, i need them to reimplement my <software> on libsysactivity" ? Boom, you have to break the ABI.

Another year and another big release. This bug is growing old!

Version 0.6.0 comes with full support for DragonFly and MANY other features. Please, somebody take a look to this bug!

Hi Carlos, 
  Have you addressed the C Struct issue? If so point me to the docs I'll take a look and make this project a reality.

Nope. I've been thinking about it a lot but I can't see an alternative to the C-structs good enough to be used.

I think at least adding a few reserved fields to each struct would be worth it. At least that way you can always have something like the following

struct x 
{
   ...
   void *reserved;
}

Then you find fields that are required and you get:

structv2
{
  Something indicating version.
  more info here
}

struct x 
{
 ....
 v2struct *ptr;
}

The ABI at least is consistent between versions even if the api has changed. I also note the lack of windows support. Is this planned? 

Finally in order for this lib to become mainstream there has to at least be one app/toolkit that makes use of it. Based on the comments from Benoît Dejean and also looking at the api's. I can't currently see a reason why this would be chosen over other libs such as libgtop. Simply put, you have to show the worth of this library. If your prepared to hack together a patch for an existing application to use it then that shows worth. Even better if you address issues that have been raised in this bug report (ie make it just as useful as libgtop and also easier to solve the issues in ksysguardd you have a very valuable project). Until that point this project can't be accepted by fd.o. If you do achieve the above, please feel free to reopen this bug.

Of course there is an app that makes use of libsysactivity, it is sentinella! (http://sentinella.sourceforge.net/) [1]. Also i'm currently making a patch for ksysguardd so it uses the library instead of implementing the same old code again.

And, if you don't see a reason to use libsysactivity over libgtop i can tell you that at least libsysactivity is desktop-agnotic. The binary is minimal and the api is versatile.

[1] http://sentinella.git.sourceforge.net/git/gitweb.cgi?p=sentinella/sentinella;a=blob_plain;f=src/Conditions/CPU.cpp;hb=master

Ah, and i'm going to include the reserved field so i can partially solve the c struct issue.

Regards.