I’m not sure if it is entirely accurate to compare them in this way, as “Matrix” refers to simply the protocol, whereas “Signal” could refer to the applications, server, and protocol. That being said, is there any fundamental difference in how the Matrix ecosystem of federated servers, and independently developed applications compares to that of Signal that would make it less secure, overall, to use?
The most obvious security vulnerability that I can think of is that the person you are communicating with (or, conceivably, oneself, as well) is using an insecure/compromised application that may be leaking information. I would assume that the underlying encryption of the data is rather trustworthy, and the added censorship resistance of federating the servers is a big plus. However, I do wonder if there are any issues with extra metadata generation, or usage tracking that could be seen as an opsec vulnerability for an individual. Signal, somewhat famously, when subpoenaed to hand over data, can only hand over the date that the account was created, and the last time it was used. What would happen if the authorities go after a Matrix user? What information about that user would they be able to gather?
Yes, the problem with MegOLM however is that the private key is automatically exchanged between devices using the same account. I think 2-3 years ago there was an exploit allowing malicious home-server admins to add additional devices under their control and thus listen in on what appeared to be e2ee conversations to the user. This specific exploit was fixed, but the principal problems of potentially insecure exchange of private keys remains in MegOLM to this day.