code_fodder code_fodder - 3 months ago 34
YAML Question

Issues installing yaml for lua with luarocks during the c-compile

I have lua, that I isntalled using luarocks.

Edit: on a windows system

I successfully installed luasocket by downloading the luasocket-2.0.2-3.win32-x86.rock file from v202-3 from here. Its an older version, but it works.

However now I need to install yaml so I can work with yaml files. Now I am forced to install using a *.src.rock file or a .rockspec file.

I have tried:

luarocks install yaml
which gives:

D:\user\workspace\_lua>luarocks install yaml
Warning: Failed searching manifest: Failed fetching manifest for https://luarocks.org - Failed downloading https://luarocks.org/manifest - C:\Users\user\AppData\Local/LuaRocks/Cache/https___luarocks.org/manifest
Warning: Failed searching manifest: Failed fetching manifest for https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/ - Failed downloading https://raw.githubusercontent.com/rocks-moonscript-org/moonrocks-mirror/master/manifest - C:\Users\user\AppData\Local/LuaRocks/Cache/https___raw.githubusercontent.com_rocks-moonscript-org_moonrocks-mirror_master_/manifest
Warning: Failed searching manifest: Failed fetching manifest for http://luafr.org/moonrocks/ - Failed downloading http://luafr.org/moonrocks/manifest - C:\Users\user\AppData\Local/LuaRocks/Cache/http___luafr.org_moonrocks_/manifest
Warning: Failed searching manifest: Failed fetching manifest for http://luarocks.logiceditor.com/rocks - Failed downloading http://luarocks.logiceditor.com/rocks/manifest - C:\Users\user\AppData\Local/LuaRocks/Cache/http___luarocks.logiceditor.com_rocks/manifest

Error: No results matching query were found.


So, then I downloaded
yaml-1.1.2-1.src.rock
and ran the command:
luarocks install yaml-1.1.2-1.src.rock


This told me that I had dependencies yaml --> lub --> luafilesystem. so then I downloaded:


  • luafilesystem-1.6.3-2.src.rock

  • lub-1.1.0-1.src.rock

  • yaml-1.1.2-1.src.rock



And I installed them one after the other with the result:


  • luarocks install luafilesystem-1.6.3-2.src.rock
    - OK

  • luarocks install lub-1.1.0-1.src.rock
    - OK

  • luarocks install yaml-1.1.2-1.src.rock
    - FAILED, see errors below



Error log:

D:\user\downloads\luarocks-2.3.0-win32\extras>luarocks install yaml-1.1.2-1.src.rock
Using yaml-1.1.2-1.src.rock... switching to 'build' mode
mingw32-gcc -O2 -c -o src/api.o -ID:/user/install/lua/luarocks/include src/api.c -Isrc
mingw32-gcc -O2 -c -o src/b64.o -ID:/user/install/lua/luarocks/include src/b64.c -Isrc
mingw32-gcc -O2 -c -o src/dumper.o -ID:/user/install/lua/luarocks/include src/dumper.c -Isrc
mingw32-gcc -O2 -c -o src/emitter.o -ID:/user/install/lua/luarocks/include src/emitter.c -Isrc
mingw32-gcc -O2 -c -o src/loader.o -ID:/user/install/lua/luarocks/include src/loader.c -Isrc
mingw32-gcc -O2 -c -o src/lyaml.o -ID:/user/install/lua/luarocks/include src/lyaml.c -Isrc
mingw32-gcc -O2 -c -o src/parser.o -ID:/user/install/lua/luarocks/include src/parser.c -Isrc
mingw32-gcc -O2 -c -o src/reader.o -ID:/user/install/lua/luarocks/include src/reader.c -Isrc
mingw32-gcc -O2 -c -o src/scanner.o -ID:/user/install/lua/luarocks/include src/scanner.c -Isrc
mingw32-gcc -O2 -c -o src/strtod.o -ID:/user/install/lua/luarocks/include src/strtod.c -Isrc
In file included from src/strtod.c:23:0:
src/strtod.c:24:17: warning: '_errno' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
extern int errno;
^
mingw32-gcc -O2 -c -o src/writer.o -ID:/user/install/lua/luarocks/include src/writer.c -Isrc
mingw32-gcc -shared -o yaml/core.dll src/api.o src/b64.o src/dumper.o src/emitter.o src/loader.o src/lyaml.o src/parser.o src/reader.o src/scanner.o src/strtod.o src/writer.o D:/user/install/lua/luarocks/lua5.1.lib -lMSVCR80
src/api.o:api.c:(.text+0x395): undefined reference to '_imp__yaml_realloc'
src/api.o:api.c:(.text+0x412): undefined reference to '_imp__yaml_string_extend'
:
LOTS OF THESE
:
src/scanner.o:scanner.c:(.text.unlikely+0x29d0): undefined reference to '_imp__yaml_queue_extend'
src/scanner.o:scanner.c:(.text.unlikely+0x29e8): undefined reference to '_imp__yaml_token_delete'
collect2.exe: error: ld returned 1 exit status

Error: Build error: Failed compiling module yaml/core.dll


Does anyone know why this build fails? - I got it directly from here:
https://luarocks.org/modules/gaspard/yaml/1.1.2-1
which is the most downloaded version... so I guess it should work?

I mean, I can see what the error is... but I don't understand why there should be an error. How can I resolve this?... Any help appreciated :)

UPDATE

Thanks to siffiejoe's comment, I was able to compile the yaml rock. Here is the output:

D:\user\downloads\luarocks-2.3.0-win32\extras>luarocks install yaml-1.1.2-1.src.rock CC="gcc -DYAML_DECLARE_STATIC"
Using yaml-1.1.2-1.src.rock... switching to 'build' mode
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/api.o -ID:/user/install/lua/luarocks/include src/api.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/b64.o -ID:/user/install/lua/luarocks/include src/b64.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/dumper.o -ID:/user/install/lua/luarocks/include src/dumper.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/emitter.o -ID:/user/install/lua/luarocks/include src/emitter.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/loader.o -ID:/user/install/lua/luarocks/include src/loader.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/lyaml.o -ID:/user/install/lua/luarocks/include src/lyaml.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/parser.o -ID:/user/install/lua/luarocks/include src/parser.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/reader.o -ID:/user/install/lua/luarocks/include src/reader.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/scanner.o -ID:/user/install/lua/luarocks/include src/scanner.c -Isrc
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/strtod.o -ID:/user/install/lua/luarocks/include src/strtod.c -Isrc
In file included from src/strtod.c:23:0:
src/strtod.c:24:17: warning: '_errno' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
extern int errno;
^
gcc -DYAML_DECLARE_STATIC -O2 -c -o src/writer.o -ID:/user/install/lua/luarocks/include src/writer.c -Isrc
mingw32-gcc -shared -o yaml/core.dll src/api.o src/b64.o src/dumper.o src/emitter.o src/loader.o src/lyaml.o src/parser.o src/reader.o src/scanner.o src/strtod.o src/writer.o D:/user/install/lua/luarocks/lua5.1.lib -lMSVCR80
Updating manifest for D:\user\install\lua\luarocks\systree/lib/luarocks/rocks
yaml 1.1.2-1 is now built and installed in D:\user\install\lua\luarocks\systree (license: MIT)


So it says it is installed. However when I try to use the yaml packge it does not find it:

local yaml = require("yaml")
local data = yaml.load("mytest.yaml")


The system reports that it can't find yaml.lua or yaml.dll. I then searched *.lua and *.dll in my luarocks folder I found:

D:\user\install\lua\luarocks\systree>dir *.lua *.dll /b /s
D:\user\install\lua\luarocks\systree\lib\lua\5.1\lfs.dll
D:\user\install\lua\luarocks\systree\lib\lua\5.1\mime\core.dll
D:\user\install\lua\luarocks\systree\lib\lua\5.1\socket\core.dll
D:\user\install\lua\luarocks\systree\lib\lua\5.1\yaml\core.dll
D:\user\install\lua\luarocks\systree\lib\luarocks\rocks\luafilesystem\1.6.3-2\tests\test.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\ltn12.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\mime.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\lub\Autoload.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\lub\Dir.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\lub\init.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\lub\Param.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\lub\Template.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket\ftp.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket\http.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket\smtp.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket\tp.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\socket\url.lua
D:\user\install\lua\luarocks\systree\share\lua\5.1\yaml\init.lua


For example, I managed to install luasocket with
luarocks install luasocket-2.0.2-3.win32-x86.rock
because there was a .rock file available. And so you can see there is a socket.lua. For yaml, lub and luafilesystem I can see various install locations, but there is no .lua or .dll files...
I searched more for yaml specifically:

D:\user\install\lua\luarocks\systree>dir *yaml* /b /s
D:\user\install\lua\luarocks\systree\lib\lua\5.1\yaml
D:\user\install\lua\luarocks\systree\lib\luarocks\rocks\yaml
D:\user\install\lua\luarocks\systree\lib\luarocks\rocks\yaml\1.1.2-1\yaml-1.1.2-1.rockspec
D:\user\install\lua\luarocks\systree\share\lua\5.1\yaml


ehh... so then I went and cloned the project from git (had to download the zip file because my stupid IT blocked my git clone ... (port 443).

So finally got that sorted, then in the yaml-master folder I tried:

luarocks make yaml-1.1.2-1.rockspec CC-"gcc -DYAML_DECLARE_STATIC"


This had the same effect as before. So now I have tried both .src.rock and .rockspec types...

I am stuck again :(

UPDATE 2

When I built from git, there was a file called yaml.lua in the top level. But all it contains is the text
yaml/init.lua
... not sure what to do with that - tried copying it into the my
..install\lua\luarocks\systree\share\lua\5.1
dir, but when I run my test lua projam it gives an error.... what is going on, it can't be this hard to install :(

Answer

The include file src/yaml.h from the github repository contains the following preprocessor code:

#ifdef WIN32
#   if defined(YAML_DECLARE_STATIC)
#       define  YAML_DECLARE(type)  type
#   elif defined(YAML_DECLARE_EXPORT)
#       define  YAML_DECLARE(type)  __declspec(dllexport) type
#   else
#       define  YAML_DECLARE(type)  __declspec(dllimport) type
#   endif
#else
#   define  YAML_DECLARE(type)  type
#endif

That means that on Windows all yaml-related functions by default are decorated with __declspec(dllimport) (the _imp__ prefixes in the linker error messages were hints). Because of this the linker expects those functions to be in an external DLL. But the yaml code is bundled and compiled together with the Lua binding into a single DLL yaml\core.dll, and so the linker complains. The correct way to fix this is to remove all import/export-related decorations from the yaml functions, and this can be achieved by defining YAML_DECLARE_STATIC. This macro should definitely be set in the rockspec (I'll post a bug report later), but a quick and dirty solution is to set it via the CC variable on the commandline:

luarocks install yaml-1.1.2-1.src.rock CC="mingw32-gcc -DYAML_DECLARE_STATIC"

After that there should be a yaml\core.dll file and a yaml\init.lua file somewhere in your LuaRocks directory. To use the Lua yaml binding you have to make sure that a require("yaml") loads the yaml\init.lua file, and that require("yaml.core") loads yaml\core.dll. This can be achieved by adding to package.path/package.cpath (from within your Lua script) or preferably by adding to LUA_PATH/LUA_CPATH. Since the installed LuaSocket already works, you probably have LUA_PATH/LUA_CPATH mostly set up correctly, but the yaml binding uses a different approach for loading its main Lua module than LuaSocket: LuaSocket has socket.lua for require("socket"), but yaml has yaml\init.lua for require("yaml"). This approach has the advantage that any Lua submodules (e.g. yaml.a) would be in the same directory as the code for the main Lua module. Anyway, you also have to add

D:\user\install\lua\luarocks\systree\share\lua\5.1\?\init.lua

to your LUA_PATH environment variable.

Btw., the yaml.lua file you found in the github repository is a symbolic link and won't work correctly on Windows anyway (but it could interfere, so you shouldn't put it into your rocks tree). It's only there for convenience during development.