Showing posts with label Solaris. Show all posts
Showing posts with label Solaris. Show all posts

Monday, June 18, 2007

SysAdmin mag & How to write unmaintainable code

June's issue of SysAdmin magazine, has some interesting Q&A's on Solaris. Questions on superblock, alternative methods for patching, etc. are given along with answers. It can be read here


Found this hilarious take on unmaintainable code written some time back when it was slashdotted:

Saturday, June 9, 2007

Core Solaris kernel paths

Sometime back I was searching online the path for core kernel binaries for Solaris but the information was hard to find and it was not exhaustive. Finally I was able to find the information in Solaris Internals book. The paths for core Solaris kernel binaries are:

/kernel/genunix - Platform independent core kernel for non-UltraSparc based systems resides in this binary. All non-UltraSparc based systems load this genunix during boot time.

/platform/sun4u/kernel/genunix - optimized binary for UltraSparc, but it is independent of the system type. This binary is loaded during boot time only by UltraSparc systems.

/platform/{arch}/kernel/unix - Platform dependent component of the core kernel resides here. {arch} is the architecture of the system.

The other kernel modules get loaded on demand later i.e. when an application requires them. They reside under the /usr directory tree.

All these binaries contain various low level kernel services that are needed to run the system. The command to find all the kernel modules in a system is

# modinfo

It will give as output the loaded modules in a running system.

Friday, June 8, 2007

Going from JDS to CDE

I have started using CDE now for Solaris. JDS was becoming a pain with its sluggish pace. It was eating a lot of memory, too. CDE seems lightweight in this regard as compared to JDS. There are a lot of things in CDE that I wish were more JDS-like. I will try to configure and see how friendly I can make it for general pupose use. It will be used for web-browsing and mails.

The main difference between CDE and JDS when one begins using CDE after a long time with other desktops is how the minimize windows feature works, and icons on the desktop that are absent in CDE. On minimizing any window, it appears as an icon on the desktop, unlike JDS's "go to bottom panel behavior."

The responsiveness of CDE is much better than JDS though and that is what sets it apart.

Tuesday, June 5, 2007

Creating a dynamic library - example

We all use library functions in the programs we write. An example of library that is always used in Solaris and Unix like Operating systems is libc.so. But how to create a library? It is not hard. A dynamic library can be easily created as shown in the following example.

Let's say we want to create a library called libgeek.so. It will contain an example function called my_library_func() that we will use in our program. We will create a simple program called geek.c that has the function we wanted. We will compile this as a library and call it libgeek.so (library names begin with lib) :

$ cat geek.c

my_library_func()
{
printf("Inside my library function");
}

The above is a library function we wanted to create. We then compile it into a dynamic library by giving a -G option to compiler :

$ cc -o libgeek.so -G geek.c

Now, we can use the generated library libgeek.so in our programs like:

$ cat hellolibrary.c

int main()
{
my_library_func();
return 0;
}

Now, we can compile our program and tell the linker to link to the library we created for my_library_func() :

$ cc hellolibrary.c -L/home/osgeek -R/home/osgeek -lgeek

L and R tell linker the path to look up during link-time and run-time to find library libgeek.so. The library libgeek.so is used with "lib" part removed and "l" prefixed as "lgeek".

When we run this program, the output would look like:

$ a.out
Inside my library function

That's it. We created a library and used it in a program.

Sunday, May 27, 2007

No archives (*.a ) in Solaris anymore

While discussing static libraries in one of my previous posts, I commented that libm is provided as both dynamic library ( libm.so ) as well as static archive ( libm.a ).

Well, that is not true for Solaris anymore. Solaris 10 doesn't ship with a single static library.
Doing
ls -la |grep *.a
in /usr/lib where libraries are usually present returned no results. Tried in some more directories with same result.
I don't know when static libraries were dropped from Solaris. My guess is that it was Solaris 10, but any pointers to information would be welcome.

Saturday, May 19, 2007

Static linking : library options in command line

In my last post I asked why it's advised that library options be the last in the command line in case of static linking.
Here is the explanation:
The symbols on the command line are resolved from left to right.
Stating linking looks through the static library for "undefined" symbols when it is processed.
Now in case of

cc -lfoo hello.c

there are no undefined symbols when libfoo.a gets processed and so nothing gets extracted from it. When the object file is processed, it doesn't find any symbol and it gives an error "Undefined symbol"
If hello.c is put before -lfoo as in

cc hello.c -lfoo

there are undefined symbols when libfoo gets processed and so they get extracted. This works fine.

Dynamic linking doesn't have this issue as all symbols are available through the virtual address space of the output file.
Static libraries have other issues like bigger executable size, and lack of ABI ( application needs to be relinked with each new version of the library).
One advantage of having static libraries is that the executables linked to them are somewhat faster at runtime because all the linking occurs before loadtime. This helps in benchmarking. Math library libm is provided as a shared object (libm.so) as well as static library (archive libm.a) since benchmarking makes a heavy use of this library.

Friday, May 18, 2007

quirk of static linking

A question related to linking today.
Why is it advised to put the library options at the end of command line for compilation?

Hint: If we have a static library, say libfoo.a which we want to link to our program hello.c

cc hello.c -lfoo
rather than
cc -lfoo hello.c

-l option tells the compiler to link to library [lib]foo. Note that "lib" from libfoo is dropped and only "foo" part is given with -l.

How Nerdy are you?

Took this Nerd test and was worried I was going to score a "less nerdy" type. But the result were pleasantly surprising. It said " All hail the monstrous nerd. You are by far the SUPREME NERD GOD!!!"
Perhaps, my score was helped by a few dirty clothes in my room, and Solaris.
For now, I am in heaven!


I am nerdier than 95% of all people. Are you a nerd? Click here to find out!

Wednesday, January 3, 2007

Memory Overcommit and the OOM Killer

Linux has a feature called memory overcommit. Put simply, it means kernel allocates memory even if it doesn't have enough. This happens when a new process is created using fork(). This effectively copies the parent's address space, and so requires twice the parent process' memory once the new process (child) is created. The memory overcommit feature means that fork() always returns a success. Even if there is not enough memory to create a new child process!
The idea behind a memory overcommit feature of Linux is that the child process rarely uses all the memory allocated to it. fork() is followed by exec() which overlays the child address space with some exectutable. Once the exec() is done, the child process exits and the parent process (which goes into wait() after creation of child) resumes.
Failing to allocate enough memory when it is needed by the child results in another process being invoked. This process is called Out Of Memory (OOM) killer. The job of this process is to select a process to kill so that the memory requirements after fork() can be satisfied. Not a very desirable feature, but it is necessary to keep memory overcommit feature of Linux. This made OOM killer infamous. How to select a process to kill is tricky. It might happen that some important processes (e.g. a database) gets killed by OOM killer. Analogies like this show how serious the situation is when killer is invoked.
It seems that during 2.4, OOM killer's favourite process to kill was the Netscape browser. The browser would crash all of a sudden and you'd have no idea why.
The memory overcommit along with OOM is not an example of a good design feature, but has even made its way into AIX. With 2.6 the memory overcommit feature can be suppressed using some variables, but by default the feature is present.
Fortunately, it doesn't exist in Solaris. Solaris never used memory overcommit. First it was vfork() instead of fork() to prevent the failure of process creation. In Solaris 10, posix_spawn() is used instead of vfork() since vfork() is not MT-safe.

Thursday, December 21, 2006

Outstanding OpenSolaris questions by James McGovern

James left a comment to one of my earlier posts and suggested I answer some of the outstanding questions he posted on his blog sometimes back. Though I am not an authority on this, I will try to answer some of them as per my understanding. I was thinking of replying in comments section but it became too long, so here is a reply to James comment.

Hi James,
Unfortunately I have little information related to SPARC chips. It's an open architecture and anyone can see the specification and is free to implement.
I don't think Sun produces SPARC chips for appliances as they are a server-focussed company.
Maybe Fujitsu does it. I have heard of SPARC chips in some cameras, but you'd have to google search to find out more.
Regarding OpenSolaris, I believe OpenSolaris.org community is much more capable to answer those queries. e.g. I searched Xen community list there and it seems they have some working Xen code for OpenSolaris. Of course, Xen itself is not yet complete, so Xen for OpenSolaris would take time. Looking at the activity there, it seems Xen is the future of OpenSolaris virtualization.
Headless/Diskless clients under Solaris have been supported for quite sometime.
About legal implications of running OpenSolaris, I know none that exist. You are free to distribute your product with an OpenSolaris distribution as long as the existing files you have used from the community and modified are open sourced under CDDL. If you've added any new files, you are free to choose whatever license you want for your files if that license permits it. CDDL is less viral in this regard.
That's my understanding. I'd suggest to throw these questions to OpenSolaris list. They'd sure give you detailed and authentic reply.

Sunday, December 17, 2006

GNU/Solaris ?

Sometimes back, The Register had an article titled "Is 'GNU/Solaris' emerging from Microsoft-Novell deal?"
But GNU/Solaris is already there, even with OpenSolaris under CDDL which is not GPL but another open source license. Maybe the reporter didn't do the homework right! Or perhaps he meant something else when he said GNU/Solaris.

ZFS in Mac OS X ?

Seems it wasn't just a rumour. ZFS is going to be in the upcoming MacOS X! Very cool to know ZFS is being ported to other OSs. It is already being ported to FreeBSD, along with DTrace. Porting ZFS to other OS is good for them as well as Solaris and Operating systems in general. It gives more visibility to such great technologies and innovation that they rightly deserve. It also gets other OS users to experience and use such powerful stuff. That would definitely attract more users to Solaris also, mainly those who still have to know how different Solaris 10 and OpenSolaris are to prior releases, and their capabilities.
The story and screenshot of ZFS in OS X was first broken here:
http://mac4ever.com/news/27485/zettabyte_sur_leopard/

Some blogs discussing it are at:
http://loop.worldofapple.com/archives/2006/12/17/zfs-file-system-makes-it-to-mac-os-x-leopard/
http://rom.feria.name/blog/2006/12/17/zfs-on-mac-os-x-105/
http://colindw.blogspot.com/2006/12/w00t-zfs-on-leopard.html
http://www.c0t0d0s0.eu/archives/2406-Its-official-ZFS-in-Leopard.html

Thursday, December 14, 2006

Live Upgrade

Solaris OS has a pretty cool technology if we want to upgrade our computer to some later release of the OS. It is called Live Upgrade. It basically works like this:
When you first install Solaris on your computer, you leave some disk space free for the future. It is not a problem since disks are cheap now. Only thing is to remember to set aside some space during first installation. When at some later time a new release of the operating system comes up and you want to install it without having to shut down your system, you can use Live Upgrade. It basically is really Live Upgrade. No downtime while upgrading. Now how many OSs have such cool stuff!
Ok so you are ready to upgrade. You just make a copy of your existing operating system boot image. It's just a command away and the empty disk space has the copy of existing Solaris. Another command and the copy gets upgraded to whatever newer release you have. Once the upgrade is over, simply set the newly upgraded space as the boot option and just one reboot after this you are running the latest bits of the OS. See? The downtime is just one reboot. All the time the system was upgrading you were using the system while the upgrade was going in the background. It just made your system a bit slower, that's all!
Though an individual can afford to waste a couple of hours in upgrading the system by shutting it down, data centers don't have such luxury. That's why they use Live Upgrade. The downtime when they want to use the latest OS is just one reboot time. It has an additional advantage. If for some reason the upgrade fails and you can't reboot into the newly upgraded partition, just revert back to the old working disk partition as your boot OS and it will work fine!

Tuesday, December 12, 2006

Solaris 10 update 3 released

Solaris 10 update 3 seems to have been released and is available for download. I have Solaris 10 update 2 which I installed mainly to learn the new cool ZFS (which I love, btw...thanks for making filesystem management so easy, guys!).
Time to get a hand on update 3 soon, before heavy traffic hits the download sites. Wonder if it has any new features for an individual user like me. It'll have bug fixes so worth getting it.
Thanks to milek and uadmin for the heads up.

Monday, December 11, 2006

Microsoft Unix

It sounds funny now, but Microsoft once actually had the most widely installed Unix base. Its version of Unix was called Xenix and it was distributed in the 80's by many vendors. What happened to it since then? Well, Microsoft sold it to SCO and moved on to develop OS/2 with IBM and then Windows NT.

Wikipedia has some interesting tidbits of information for Xenix here.
How it looked like back in the 80's? Here is a screenshot from wikipedia.

System V release 4, the standard for Unix today was formed by merging SunOS, BSD, Xenix, and System V.

Of course, the legacy of Microsoft Xenix is still around. But where to look to see the history of Unix ? All the flavours of Unix are closed source, or are they? Thanks to open sourcing of Solaris, we can now take a look into all the real Unix code and find some gems of Copyrights that silently narrate the history of Unix development.

For example, to see how the development of Unix has passed on from the University of California at Berkeley to AT&T and Microsoft to Sun Microsystems, have a look at this tar code.

Such is the beauty of Unix. Decades older than any other present day OSs and still holding on its own in the modern world. Not only that, it manages to beat others often in their own game and still come out at other times with such innovations that are the envy of the youngsters. Even spawning dozens of clones which are cool in their own way. Ubuntu, anyone?

Me? I'm happy with my good ol' Unix. Solaris, that is. For me.

Sunday, December 10, 2006

MacOS X and Linux expert views , anyone

I must confess I haven't touched Linux for a long time now. The last time I seriously worked on it was RedHat 7.2 Linux which I used to learn MPI during my masters. I know it definitely has progressed a lot in the last few yrs. So, what are the new features added to the kernel or the distributions since then. I can't seem to remember any apart from some filesystems and lots of drivers.
Newer version of MacOS X will have X-Ray technology which is nothing but DTrace with a nice GUI on top.
If you know of some good features that are in Linux now, especially which are as revolutionary as the ones in Solaris ( see my last post) and are not available elsewhere, please leave a comment. I might try a new distro sometime soon.
Xen, I think would be a cool addition, but it is not unique for Linux. It will be very interesting to work with, though.

Best Operating System for geeks?

Most wannabe geeks would say Linux, though it is just a kernel and not an OS. Some would answer Ubuntu or Gentoo depending on which is the 'in thing'. A few people would perhaps say MacOS X is the best. Windows, of course, is not the one geeks want to be associated with.

One more OS is making waves in the academia and business world and with serious geeks for the last yr or so, after it was open sourced. For all the right reasons. That is the grand daddy of all, the most popular Unix - Solaris OS. No other operating system, not even Linux can claim to have as much geeky meat as Solaris. Some of the mouth watering stuff in Solaris 10, especially for geeks include:

Zones - software virtualization feature which has no match in any other OS. I can't remember which technology in any other OS comes even close. BSD jails, perhaps.

ZFS - the ultra modern file system, again with no match. The only thing that comes close is VxFS but ZFS is free with source code in the open.

DTrace - again no match anywhere. The capability to look into each and every place into the kernel and other parts in a running system using DTrace is unparalleled. It has been winning accolades all over. SystemTap for Linux is still not complete and its design makes it unlikely to be able to compete with DTrace in the future.

BrandZ - It gives you the capability to run Linux apps on top of Solaris. For example, you can run a version of Linux like CentOS right inside a Solaris Zone. Say you want to play Quake or use Google Earth which don't have Solaris apps; just create a Zone in Solaris, install your Linux in the zone and play away with the apps which are available in Linux. How more geeky can one get!

Of course, serious business won't play games. For them there are other more serious features like:
Fault Management Architecture (FMA), Service Management Framework(SMF), etc.

Linux is a good Operating System and has the advantage of having more drivers. But in almost everything else, Solaris scores higher. Now that it is open sourced it should get people interested in creating drivers. The community and codebase of Solaris is called OpenSolaris. Looking at the number of posts and projects there, it really looks like a dynamic and vibrant group of geeks.

Steps to install PyTorch on VMware workstation (Ubuntu guest)

  The following is the list of steps to install pytorch 2.0 in VMware workstation (Ubuntu guest): $ mkdir ~/pytorch $ mkdir ~/pytorch/as...