SCons libraries and sub-libraries

scons link static library
scons sharedlibrary
scons include library
scons variantdir
scons sources
the scons foundation
scons sharedobject
scons unix

I have a hierarchical build system based on SCons. I have a root SConstruct that calls into a SConscript that builds a shared library and then into a different SConscript that builds an executable that depends on the shared library.

So here's my question: my understanding of shared libraries on linux is that when you want to do the final ld link for the executable that will be using the shared lib, the shared lib has to be included on the executable's ld command line as a source to reference it (unless it's in a standard location in which case the -l option works).

So here's something like what my SCons files look like:

=== rootdir/SConstruct

env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env.Append( LIBS=[shared_lib] )
executable = SConscript('barexec/SConscript')

=== rootdir/foolib/SConscript

env=DefaultEnvironment()
env.Append(CPPPATH=Glob('inc'))
penv = env.Clone()
penv.Append(CPPPATH=Glob('internal/inc'))
lib = penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c']
Return("lib")

=== rootdir/barexec/SConscript

env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )
Return("exe")

So the hitch here is this line:

env.Append( LIBS=[shared_lib] )

This would be a great way to add generated libraries to the command line for any other libs that need them, EXCEPT that because SCons is doing a two-pass run through the SConscripts (first to generate it's dependency tree, then to do the work), rootdir/foolib/libfoo.so winds up on the command line for ALL products, EVEN libfoo.so itself:

gcc -g -Wall -Werror -o libfoo.so foo.o morefoo.o libfoo.so

So how is this best done with SCons? For now I've resorted to this hack:

=== rootdir/SConstruct

env=DefaultEnvironment()
shared_lib = SConscript('foolib/SConscript')
env['shared_lib'] = shared_lib
executable = SConscript('barexec/SConscript')

...

=== rootdir/barexec/SConscript

env=DefaultEnvironment()
exe = env.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] + env['shared_lib'] )
Return("exe")

Is there a more SCons-y way of doing this?

You should allow the shared libraries to be found by the build.

Look for the LIBPATH and RPATH variables in the SCons documentation; these are the "Scons-y" way to set up search paths so that any generated -l options find libraries properly.

Having mentioned the above, here's what you should see gcc do based on the setup of SCons (and if it doesn't, you may have to do it manually).

The -l option always finds shared libraries provided that you also give the compiler the location of the library. There are two times this is needed: at compile time (-L option) and at runtime (-rpath generated linker option).

The LIBPATH SCons setup should generate something that looks like -L/some/directory/path for the compile-time search path.

The RPATH SCons setup should generate a linker option to embed a search path; e.g. -Wl,-rpath -Wl,\$ORIGIN/../lib would embed a search path that searches relative to the executable so that executables placed in bin search in the parallel lib directory of the installation.

SCons User Guide 2.1.0, SCons makes it easy to create libraries and to use them in the programs. Building Libraries. You build your own libraries by specifying Library instead of Program:� This post introduces a small enhancement to my SCons shortcuts system – nicer support for external libraries via the with_libs keyword argument. In recent episodes , I needed to link with the protobuf library to use the Protocol-Buffers-based AddressBook library .

Here is a better way to organize your SConsctruct/SConscript files. Usually with Hierarchical builds you should share the env with the rest of the sub-directories. Notice that I cloned the main env in the barexec directory as well, so that the foolib is only used to link that binary.

=== rootdir/SConstruct

import os

env=DefaultEnvironment()

subdirs = [
    'foolib',
    'barexec'
]

# The exports attribute allows you to pass variables to the subdir SConscripts
for dir in subdirs:
    SConscript( os.path.join(dir, 'SConscript'), exports = ['env'])

=== rootdir/foolib/SConscript

# inports the env created in the root SConstruct
#
# Any changes made to 'env' here will be reflected in
# the root/SConstruct and in the barexec/SConscript
#
Import('env')

# Adding this 'inc' dir to the include path for all users of this 'env'
env.Append(CPPPATH=Glob('inc'))

penv = env.Clone()
# Adding this include only for targets built with penv
penv.Append(CPPPATH=Glob('internal/inc'))
penv.SharedLibrary( 'foo', source=['foo.c', 'morefoo.c'])

=== rootdir/barexec/SConscript

Import('env')

clonedEnv = env.Clone()

# The foo lib will only be used for targets compiled with the clonedEnv env
# Notice that specifying '#' in a path means relative to the root SConstruct
# for each [item] in LIBS, you will get -llib on the compilation line
# for each [item] in LIBPATH, you will get -Lpath on the compilation line
clonedEnv.Append(LIBS=['foo'], LIBPATH=['#foolib'])

clonedEnv.Program( 'bar', source=['main.c', 'bar.c', 'rod.c'] )

Building and Linking with Libraries, Building Shared (DLL) Libraries: the SharedLibrary Builder For python2 It's important to note when creating tools within sub-directories, there needs to be a� :When adding google play-services library so I can add in a map I got this error: all com.android.support libraries must use the exact same version specificationMy dependencies are as follows:dependencies { implementation fileTree(dir: 'libs', inclu

Additional to Brady decision i use static/global variables to store targets name and path. It's allow me more control over build.

# site_scons/project.py
class Project:
  APP1_NAME = "app1_name"
  APP2_NAME = "app2_name"
  MYLIB1_NAME = "mylib1_name"
  # etc
  APP_PATH = "#build/$BuildMode/bin" # BuildMode - commonly in my projects debug or release, `#` - root of project dir
  LIB_PATH = "#build/$BuildMode/lib"
  @staticmethod
  def appPath(name) :
     return os.path.join(APP_PATH, name)
  @staticmethod
  def libPath(name) :
     return os.path.join(LIB_PATH, name)

Define targets:

from project import Project
...
env.SharedLibrary(Project.libPath(Project.MYLIB1_NAME), source=['foo.c', 'morefoo.c'])

Application:

from project import Project
...
env.Append(LIBPATH = [Project.LIB_PATH])
env.Append(LIBS = [Project.MYLIB1_NAME])
env.Program(Project.appPath(Project.MYAPP1_NAME), source=[...])

In my projects it works fine, scons automatically find depends of library without any additional commands. And if i want to change name of library i just change my Project class.

SCons 3.1.1, name of sub.dir will generate a corresponding sub/dir/Foo.class class file. Library(). A synonym for the StaticLibrary builder method. LoadableModule() , env. i am trying to develop cross platform app ( on native level ) that uses POCO socket library with android NDK. but i dont know how to integrate/use POCO library in android NDK using gradle in studio.i am using com.android.tools.build:gradle-

Builders, Building Static Libraries Explicitly: the StaticLibrary Builder; Building Shared (DLL ) classes = Java(target = 'classes', source = 'src/pkg/sub') JavaH(target� Storage and data persistance for Parrot VM. Contribute to Whiteknight/ParrotStore development by creating an account on GitHub.

SCons User Guide 0.96.93, Building Shared (DLL) Libraries: the SharedLibrary Builder For python2 It's important to note when creating tools within sub-directories, there needs to be a� This is what precludes "libraries" from using other "libraries". A better s/w build methodology would have built all the libraries as libraries (not just the ones you "think" are being used) then added all the "libraries" directories to the include path rather than adding an include path for each library.

SCons 4.0.1, Building Shared (DLL) Libraries: the SharedLibrary Builder. 4.2. And then in your SConscript or any sub-SConscript anywhere in your build, you can import� > > simple: all libraries it wants are in sources or <library> > > properties. Note that today, there only two "dependency" features: > > <dependency> and <library>. The first is mostly used to get at > > usage-requirements. The second is used to *build* the specified > > library and link it to the application. Unfortunately, the first

SCons User Guide 1.2.0, Building Shared (DLL) Libraries: the SharedLibrary Builder. 4.2. Linking with Libraries scons -Q ['foo.c', 'sub/dir/value'] scons: `.' is up to date. There is also a � Files hierarchy in Visual Studio. Sort projects inside the solution. This is the part that I had more trouble finding on Internet. There seems to be pieces everywhere, but the documentation itself was not clear enough for me, so here we go.

Comments
  • This sounds like a good idea, but doesn't this subvert SCons' dependency checking? If I add LIBS=foo to the bar program environment, then SCons won't necessarily know where gcc is getting libfoo.so (could be my project, could be /usr/lib or whatever as far as SCons knows), and won't detect the dependency between libfoo.so and barexec. I'd have to hack a dependency between foo and bar with env.Depends() wouldn't I?
  • @TedMiddleton, you have to tell SCons which libraries an executable needs (via the LIBS construction env), and where they are (via the LIBPATH construction env), then the dependency tree is created based on this. SCons will indeed know about them since it has these variables, which it subsequently passes to gcc or whichever compiler. And you will not need to hack any dependencies with the env.Depends() function. Try it and you'll see :)
  • Something I discovered playing around with this after asking just this question is that DefaultEnvironment() accesses a global variable across SConscripts and sub-SConscripts, so unless you want to pass down a different environment, Import() and Export() aren't strictly necessary.
  • Also, see my comment above to Kevin Grant about LIBS=foo - does SCons detect the build dependency between libfoo.so and barexec if you do this?
  • @TedMiddleton, I do this sort of env cloning in my build system. The different environments are indeed isolated with respect to their variables etc, but the targets (libfoo.so and barexec) are still "detected" doing it this way. Im not certain, but I think the targets are global, they must be for this to work, and it does indeed work :)
  • $LIBPATH shouldn't be evaluated until the dependencies are walked (after all SConscript/SConstruct evaluation). So the order of barexec and foolib shouldn't matter.
  • @bdbaddog I also expected the order of subdirectories to be irrelevant, but in fact when I switch the order in the example above, the linker is invoked without the -L switch (SCons 3.0.1) – do you have any idea why that's the case? (I just fixed some other errors in my initial answer)
  • Add print("LIBPATH=%s"%clonedEnv['LIBPATH']) and paste the output above. (Right after your Append(LIBPATH..))
  • @bdbaddog it prints LIBPATH=['$FOOLIBDIR'] in both cases – but I noticed something else: when foolib is before barexec and I use --debug=tree, I see a dependency .debugdebug/barexecdebug/barexec/bardebug/foolib/libfoo.dylib. When I change the order, foolib is missing from barexec's dependencies! As a result, SCons builds foolib after barexec and $FOOLIBDIR is only set after barexec has been built. What's the correct solution for this?
  • Ahh.. The issue is your env.Clone()'s in the subdir SConscripts. Specifically barexec's. See latest push to github. So if you have barexec's SConscript first it Clones() the imported env before FOOLIBDIR is set in the "global" env which is passed around. So when it's actually evaluated there is no FOOLIBDIR in the cloned "clonedEnv". In these examples no need to clone. You can specialize LIBPATH and LIBS in the builder call.