I have been using linuxbrew for a while, the reason being not having root access to install the packages I love.

Installing linuxbrew on centos 6

Linuxbrew on centos 6 is tricky because centos 6 is old OS. Linuxbrew wiki had steps to do this. Sadly these steps don’t work now. So the hack to get it working was creating symlink of modern (version 5+) gcc and g++ in ~/.linuxbrew/bin. This seems to do the trick.

Building Halide using linuxbrew compiler

I was building halide with the gcc installed via linuxbrew. The built halide blur example wasn’t working. Now halide runtime doesn’t even dynamically links with cuda. Instead it loads important cuda function pointers using dlopen and dlsym. After building debug halide runtime, turned out the cuInit api call was failing with return status -1. So to debug a simpler example, I created smaller example:

#include <dlfcn.h>
#include <stdio.h>
#include <assert.h>
int main(int argc, char **argv) {
  void *handle = dlopen("libcuda.so", RTLD_LAZY);
  int (*cuInit)(unsigned int);
  if (!handle) {
    fprintf(stderr, "%s\n", dlerror());
    return 1;
  }
  dlerror();    /* Clear any existing error */

  cuInit = (int (*)(unsigned int)) dlsym(handle, "cuInit");
  char *error = dlerror();
  if (error != NULL) {
    printf("%s\n", error);
    return 1;
  }
  printf("return val: %d\n", (*cuInit)(0));
  assert((*cuInit)(0) == 0);
  return 0;
}

The example when compiled with system g++, the example worked without hitch, but with linuxbrew gcc it didn’t work! The difference in the binaries was, the RPATH elf field was set to ~/.linuxbrew/lib in the binary generated by linuxbrew g++.

Linuxbrew uses different dynamic linker than system’s

So because RPATH was set and the RPATH contained ld.so. It was using different dynamic linker than systems. The linuxbrew ld.so doesn’t seem to have default search path configuration and it doesn’t search libraries in /usr/lib64. Setting explicit LD_LIBRARY_PATH does the trick! So running:

LD_LIBRARY_PATH=/usr/lib64:$LD_LIBRARY_PATH nvprof ./bin/blur_test

confirms that the example indeed runs successfully on GPU 🙂.

Happy Coding!.