x-tags (the basis for Brick) and Polymer use exactly the same polyfills, but x-tags uses a reduced set.
Brick itself ships as a single minified JS file, so that distribution doesn't work with Polymer (if you load
platform.js, there are polyfill collisions).
However, the x-tags themselves can work with Polymer. If you load
platform.js (the polyfills), then
x-tag-core (x-tag's sugaring layer), then you can import Polymer elements and load x-tags together with no problems.
This process can be simplified using imports, but x-tags/brick doesn't ship with supporting imports files (yet, but we are trying to work that out). There's an article at http://www.polymer-project.org/articles/polymer-xtag-vanilla.html with more information and some examples.