I am compiling some code but, I am having problems with the path to custom header files.
Here is the structure of the repository.
An R package compilation is always relative to
src/ which becomes the working directory for the build.
If you want to add that directory explicitly you can add
PKG_CXXFLAGS = -I.
And if you want to add the directory
inst/include/ you would have to add
PKG_CXXFLAGS = -I../inst/include
as you need to be relative to
src/. You can combine both.
As for your SeqLib sources: you can move things around, or adjust your
src/Makevars as needed.