I have a C++ method made Q_INVOKABLE. I can call this method from QML and it works when it returns basic types (like QString). But I can't with a custom type. How should I do this? Should I return a QVariant instead? Ideally, I would like to return a pointer to my custom type if possible.
qmlRegisterType<MyType>("Mine", 1, 0, "MyType");
qmlEngine->rootContext()->setContextProperty("testObj", new MyType());
QVariant is the way to go for your custom class in that construction. Make sure you register your class.
That being said, you may wish to consider using Q_PROPERTY instead, for your custom type and then you can access that even without a function call. If you need custom parameters to the method and you cannot rearrange the code, this is obviously not an option.