One way that we monitor API calls from within our customer’s applications is through our agent. The Bearer Agent hooks into every API call in order to read the request, read the response, and in some scenarios act upon that information.
The agent replaces methods in the HTTP clients with instrumented versions that call the original methods. Each programming language supports different ways of handling this, but for Ruby it is done either by using method aliases with the alias_method or by prepending a module with module#prepend. Unfortunately, these approaches are incompatible with one another and there is no consensus amongst libraries as to which approach to use. This means that there can be conflicts between different gems that you're using in your application, this causes stack level too deep errors at run time.
The Bearer agent attempts to detect these errors and provide a more helpful error. In this case, an exception of type BearerAgent::Interceptor::ConflictDetector::Error will be raised.
Examining each override technique
Before looking at each implementation, let’s assume we have a basic HTTP client gem with a class like this:
This client has a single method that accepts a URL and performs the necessary actions to make an HTTP GET request.
The aliasing approach
Method aliasing with `alias_method` makes a copy of the original method that you can then call again later. You might use it in instances where you need to make modifications to the method, but still want to retain the original, like in our agent use case. With method aliasing, we re-open the class in another piece of code and replace the get method as follows:
In the code above, the new get performs some additional actions before calling the original method. Now every time the consumer of HTTPClient calls the get method, it first runs our custom code before running the original get code.
In this example, we only call puts to display a message, but in practical usage we define our custom get with all of our logging and monitoring logic so it occurs in this override.
A downside to this approach is that errors raised in the instrumented code will be displayed as having occurred in the file containing the original class. In other words, blame gets placed on the original client rather than our addition to it.
The module prepending approach
With module prepending, we create a separate module with the instrumented method and insert it into the inheritance chain of the class as follows:
In this instance, when the consumer calls HTTPClient’s get, Ruby first looks to our definition of get before executing the original—where we’ve used super.
This is the default approach we use with the Bearer Agent. It has an advantage over method aliasing as errors that occur in the instrumented code will be correctly attributed to the file containing the instrumented code.
The incompatibility problem
A problem arises when a method is hooked initially by using the prepend approach, and then later by using the alias approach.
When a call to get is made, we see the following output:
The call to super in the "prepend hook" method calls the "alias hook" method, but because the get_without_other_gem alias was taken after the module was prepended, it ends up calling the other hook again rather than the original method as we expect. This means we end up in a loop where each approach calls the other one without ever completing the call to the original get.
Working around the issue
There are two ways to work around this issue. The first is to ensure that any code that uses the prepend approach is called after any code that introduces aliases. This makes it so the aliases refer to the original unpatched method rather than the prepended hook. In some cases, this can be achieved by reordering gems in the Gemfile or by reordering require statements in your code. This still isn’t ideal because in some cases the hooks may not be applied until an initializer is invoked (the Bearer agent works in this way), or when some event occurs in the framework you are using. It can be difficult to work out how to set up your application correctly to make sure that everything happens in the right order.
The other option is to ensure all gems hooking a particular method consistently use prepending or aliasing, but not a mix of the two. Unfortunately, many gems do not provide a way of choosing the approach and it will take digging into their source to confirm which technique is used.
Our solution was to provide a hook_mode configuration as part of the Bearer Agent. The agent uses prepending by default, but may be configured to use aliasing during initialization as follows:
For most users, this won’t be necessary, but if you do find yourself seeing the BearerAgent::Interceptor::ConflictDetector::Error error when adding the Bearer Agent then the hooks_mode option is right for you. As the community still lacks a preferred choice between aliasing and module prepending, we'd love to see more libraries offer their users a choice to avoid compatibility conflicts.