Exagon Exagon - 1 year ago 104
C++ Question

strange behaviour of parser with boost spirit x3

I have a parser for parsing an Identifier like

foo, bar, baz
and one for parsing also nested identifiers like
foo::bar, foo::bar.baz, foo::bar.baz.baham

They both parse into the same ast struct, which looks like this:

struct identifier : x3::position_tagged{
std::vector <std::string> namespaces;
std::vector <std::string> classes;
std::string identifier;


The parser for an
looks like this:

#define VEC_ATR x3::attr(std::vector<std::string>({})) //ugly hack

auto const identifier_def =
>> id_string;

and for the
like this:

auto const nested_identifier_def =
(+(id_string >> "::") >> +(id_string >> ".") > id_string)
| (+(id_string >> "::") >> VEC_ATR > id_string)
| (VEC_ATR >> +(id_string >> ".") > id_string)
| identifier


I know shame on me for the macro.
The identifier parser works fine, but the
has a strange behaviour
if I try to parse something like
the ast objects which falls out of the parser, has all the namespaces, in this case
twice in the
I have a small example of this strange behaviour
Can anybody explain me why this happens, and how I can avoid this?

jv_ jv_
Answer Source

The reason why you get that behaviour is that the alternative parser does not automatically rollback the changes made to the external attribute when one of its branches fails.

In your case this is what happens:

  • Initially the attribute is [{},{},""].
  • The first alternative branch is tried.
  • id_string >> "::" matches twice and adds foo and bar to the first vector ->[{foo,bar},{},""].
  • id_string >> "." fails to match -> the sequence fails -> the alternative branch fails (leaving the attribute unchanged).
  • The second alternative branch is tried.
  • id_string >> "::" matches twice and adds foo and bar to the first vector ->[{foo,bar,foo,bar},{},""].
  • attr(vector<string>({})) succeeds (attr always succeeds) and substitutes the empty second vector with a vector with an empty string -> [{foo,bar,foo,bar},{""},""].
  • id_string matches and baz is added to the attribute ->[{foo,bar,foo,bar},{""},baz].
  • The second alternative branch succeeds.

In Spirit.Qi the solution in this case is quite easy, simply use the hold directive. Unfortunately this directive is not yet implemented in Spirit.X3. A possible alternative could be putting each of the alternative branches in its own x3::rule either explicitly or with as<ast::identifier>(alternative_branch) as used here by sehe. Here is a simplified example that shows the as approach.

Another possibility could be implementing the hold directive, here is my attempt(running on WandBox):

#include <boost/spirit/home/x3/support/context.hpp>
#include <boost/spirit/home/x3/core/skip_over.hpp>
#include <boost/spirit/home/x3/core/parser.hpp>

namespace boost { namespace spirit { namespace x3
    template <typename Subject>
    struct hold_directive : unary_parser<Subject, hold_directive<Subject>>
        typedef unary_parser<Subject, hold_directive<Subject> > base_type;
        static bool const is_pass_through_unary = true;
        static bool const handles_container = Subject::handles_container;

        hold_directive(Subject const& subject)
          : base_type(subject) {}

        template <typename Iterator, typename Context
          , typename RContext, typename Attribute>
        bool parse(Iterator& first, Iterator const& last
          , Context const& context, RContext& rcontext, Attribute& attr) const
            Attribute copy(attr);
            if (this->subject.parse(first, last, context, rcontext, copy))
                traits::move_to(copy, attr);
                return true;
            return false;


    struct hold_gen
        template <typename Subject>
        hold_directive<typename extension::as_parser<Subject>::value_type>
        operator[](Subject const& subject) const
            return { as_parser(subject) };

    auto const hold = hold_gen{};
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download