Jason R Jason R - 23 days ago 14
C++ Question

Are Boost.Qi rules always held by reference inside expressions that use them?

Say I have a simple type

foo
:

struct foo { void bar(int) { // do something } };


I would like to use Boost.Qi to parse integer fields out of a string and call
foo::bar()
with the resulting values. This can be done with semantic actions, like this:

std::string str;
bar b1, b2;
boost::spirit::phrase_parse(str.begin(), str.end(),
boost::spirit::int_[boost::bind(&foo::bar, &b1, _1)] >>
boost::spirit::int_[boost::bind(&foo::bar, &b2, _1)]);


This has a lot of repetition, so I could remove some of the boilerplate like this:

template <typename Iterator>
boost::spirit::qi::rule<Iterator, int(), boost::spirit::qi::space> parse_bar(bar *b)
{
return boost::spirit::int_[boost::bind(&foo::bar, _1)];
}

std::string str;
bar b1, b2;
boost::spirit::phrase_parse(str.begin(), str.end(),
parse_bar(&b1) >> parse_bar(b2));


This looks better and it works. However, sometimes I want to save the parser expression for later (for instance if I want to capture it by value for lazy use in a lambda function). In the original example, I can do the following:

std::string str;
bar b1, b2;
auto parser = boost::proto::deep_copy(
boost::spirit::int_[boost::bind(&foo::bar, &b1, _1)] >>
boost::spirit::int_[boost::bind(&foo::bar, &b2, _1)]);
auto lambda = [parser]() { boost::spirit::phrase_parse(str.begin(), str.end(), parser); };

// some time later
lambda();


This also works (although the call to
boost::proto::deep_copy()
is needed to ensure that none of the internal nodes in the AST are held by reference). My first impression was that I could apply the same
rule
refactoring to simplify the code above to:

std::string str;
bar b1, b2;
auto parser = boost::proto::deep_copy(parse_bar(&b1) >> parse_bar(&b2));
auto lambda = [parser]() { boost::spirit::phrase_parse(str.begin(), str.end(), parser); };

// some time later
lambda();


However, this results in problems inside the later call to
lambda()
. Based on my debugging, it appears that the
rule
objects returned by
parse_bar
are always held in the expression by reference, even after the call to
deep_copy()
. Since the
rule
objects are rvalues in the line that contains the call to
deep_copy()
, the references to them are invalid during the later call to
phrase_parse()
.

This would seem to suggest that
rule
objects are always intended to be lvalues with a lifetime that at least matches the lifetime of the expressions that reference them. Is this true? I think I might be misunderstanding a key concept of the library and am trying to do this the "wrong way." In my application, I don't have formal grammars; I'm looking for an easy, compact way to define a large number of similar parser expressions inline, with semantic actions that invoke bound member functions as shown above.

Answer

Simple answer: Yes.

<!-- reads the rest of your question -->

There are many inaccuracies in your code samples.

First Snippet

  1. bar b1, b2; // probably meant foo?
  2. boost::spirit::phrase_parse doesn't exist, neither does boost::spirit::int_
  3. boost::bind is not ok as a semantic action
  4. phrase_parse takes a skipper, you provide none

Fixed:

    std::string str = "123 234";
    qi::phrase_parse(str.begin(), str.end(),
            qi::int_[boost::bind(&foo::bar, &b1, _1)] >>
            qi::int_[boost::bind(&foo::bar, &b2, _1)],
            qi::space);

Second Snippet

In the subsequent sample you have more mixup of bar and foo, you pass qi::space as a type param, bind fails to bind to b etc. etc. Without repeating the above and skipping the obvious errors

  1. there is a fundamental problem where the template function needs to be used as parse_bar<std::string::const_iterator>. I'd say it no longer looks "nice".
  2. the rule declares int() meaning you expose an attribute, but it's never assigned or used

Fixed:

template <typename Iterator>
static qi::rule<Iterator, qi::space_type> parse_bar(foo *b) {
    return qi::int_ [boost::bind(&foo::bar, b, _1)];
}

int main() {
    foo b1, b2;
    using It = std::string::const_iterator;

    std::string const str = "234 345";
    qi::phrase_parse(str.begin(), str.end(), parse_bar<It>(&b1) >> parse_bar<It>(&b2), qi::space);

Third Snippet

Mostly the same issues, besides, str is not captured.

Fourth Snippet

Yes. See the first line of my answer

Solutions

Think outside the box. You want "cute" syntax for binding a parser to an attribute out-of-band.

Note

  • This is like going back to the Spirit V1 (classic) design. It's not the spirit [sic] of the library, so to speak
  • If you want just that, use inherited attributes?

    std::string const str = "111 222";
    
    // use phoenix action 
    auto foo_bar = std::mem_fn(&foo::bar);
    px::function<decltype(foo_bar)> foo_bar_(foo_bar);
    
    // with inherited attributes
    qi::rule<It, void(foo*)> _foo = qi::int_ [ foo_bar_(qi::_r1, qi::_1) ];
    
    auto lambda = [_foo,&b1,&b2,str] { qi::phrase_parse(str.begin(), str.end(), _foo(&b1) >> _foo(&b2), qi::space); };
    
    // some time later
    lambda();
    
    std::cout << b1 << ", " << b2 << "\n";
    
  • If you want 'magical' support for your types like foo and it is logically like assignment/conversion, use the traits: http://www.boost.org/doc/libs/1_62_0/libs/spirit/doc/html/spirit/advanced/customize.html

Live Demo

I always like to add a live demo, so here you go:

Live On Wandbox

#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/bind.hpp>
#include <iostream>

namespace qi = boost::spirit::qi;
namespace px = boost::phoenix;

struct foo { 
    int value = 42;
    void bar(int i) { value = i; } 
    friend std::ostream& operator<<(std::ostream& os, foo const& f) { return os << "{" << f.value << "}"; }
}; 

template <typename Iterator>
static qi::rule<Iterator, qi::space_type> parse_bar(foo *b) {
    return qi::int_ [boost::bind(&foo::bar, b, _1)];
}

int main() {
    foo b1, b2;
    using It = std::string::const_iterator;

    // snippet 1
    {
        std::string str = "123 234";
        qi::phrase_parse(str.begin(), str.end(),
                qi::int_[boost::bind(&foo::bar, &b1, _1)] >>
                qi::int_[boost::bind(&foo::bar, &b2, _1)],
                qi::space);

        std::cout << b1 << ", " << b2 << "\n";
    }

    // snippet 2
    {
        std::string const str = "234 345";
        qi::phrase_parse(str.begin(), str.end(), parse_bar<It>(&b1) >> parse_bar<It>(&b2), qi::space);

        std::cout << b1 << ", " << b2 << "\n";
    }

    // snippet 3
    {
        std::string const str = "345 456";

        auto parser = boost::proto::deep_copy(
                qi::int_[boost::bind(&foo::bar, &b1, _1)] >>
                qi::int_[boost::bind(&foo::bar, &b2, _1)]);

        auto lambda = [parser,str]() { qi::phrase_parse(str.begin(), str.end(), parser, qi::space); };

        // some time later
        lambda();

        std::cout << b1 << ", " << b2 << "\n";
    }

    // snippet 4
    {
        std::string const str = "456 567";
        auto parser = boost::proto::deep_copy(parse_bar<It>(&b1) >> parse_bar<It>(&b2));
        auto lambda = [parser=qi::copy(parser), str]() { qi::phrase_parse(str.begin(), str.end(), parser, qi::space); };

        // some time later
        //lambda(); 
        //// no workey
    }

    // SOLUTIONS
    {
        std::string const str = "111 222";

        // use phoenix action 
        auto foo_bar = std::mem_fn(&foo::bar);
        px::function<decltype(foo_bar)> foo_bar_(foo_bar);

        // with inherited attributes
        qi::rule<It, void(foo*)> _foo = qi::int_ [ foo_bar_(qi::_r1, qi::_1) ];

        auto lambda = [_foo,&b1,&b2,str] { qi::phrase_parse(str.begin(), str.end(), _foo(&b1) >> _foo(&b2), qi::space); };

        // some time later
        lambda();

        std::cout << b1 << ", " << b2 << "\n";
    }
}