The web is a place full of standards and best practices. And then there are those unfortunate developers who has to implement something without any possibility to choose – nor the programming language,nor the server. So sometimes a system administrator has to investigate the reason why something that simply should work is simply not working and has to dig through protocols and servers and code in languages he doesn’t know because it MUST WORK BY TODAY.
So, the problem was that we had a webapp which sent mails through a provider. After a change in management the same portal had to send mails through another provider. Well, the portal is developed in .NET , it is already working, it is using standard libraries, just change all credentials and server name. It should be easy, right?
Wrong. For some reason, it isn’t working. I don’t have such an expertise in mail and related protocols, so I started to limit the boundaries of what could have been. Immediatly I found out that the server was up and running,and I could reach it both on the server and on the client (no firewall nor environment involved). In fact, using a particular combination of ports and protocols I was even able to send and email.
So likely it is a problem in the code. So I put myself to remember the little c# I once used, and studied how to use the library.
Simply put, there are two kinds of SSL encryption, which is used by SMTPS protocol: implicit and explicit. And Microsoft System.Net.Mail supports just explicit mode, as specified here: Microsoft MSDN Blog . And as always when there’s an important feature, used by a good part of the web, missing from a library, it’s “by design”.
So, what can you do? Well, you can use the fantastic (and DEPRECATED!!) System.Web.Mail . Here you can find an example of how to use it: Forum ASP.NET . Otherwise there are some other third part library, but I cannot suggest anyone of them, I didn’t tried them.
Soon after starting debugging I found out that System.Net.Mail is the standard library in C# to send emails. Maybe it is a little cumbersome and not very easy to use, but it is around by 10 years at least, so any bug should be resolved. Syntax is ok, and logic seems ok too. The behaviour is strange anyway. Let’ summarize:
– On port 25, plain old unencrypted SMTP is working – but a requirement is that mails are encrypted.
– On port 25, SMTPS (encrypted SMTP) doesn’t work; the connection timed out.
– On port 465, SMTP answers with “connection refused”. So the server it’s there, but doesn’t like the way we communicate.
– On port 465, SMTPS connection timed out.
– On port 587, any connection is refused.
Maybe you start to see the pattern. At this point I started to suspect it had something to do with encryption.
So,after some test changing encryption algorithms, I asked Mother Google if she knew anything about SMTPS in C# with the provider Aruba. A LOT of digging – it was a really difficult google fu this time – brought me in the right direction. It seemed that to send via Aruba you had to use another .NET library, the old and deprecated System.Web.Mail. Why? That was even harder to find out.
In the end I discovered that there are two kinds of SSL encryption, called IMPLICIT and EXPLICIT.
In general, in Implicit SSL the first step is always to create an encrypted communication tunnel between client and server; then begins the communication in the requested protocol. HTTPS, for example, always works this way: every message sent by client and by server is encrypted. Regarding other protocols, there’s an implicit SMTPS, an implicit FTPS and an implicit POP3S. All these protocols use a dedicated port, different from the unencripted one.
Explicit SSL works in a slightly different way. The first messages sent are on an unencrypted channel (say, classic port 21 for FTP and 25 for SMTP), then the client asks for encryption (for example, issuing the command SMARTTLS in SMTP) and finally the communication channel is switched on another port (depending on the protocol. standard or agreed during communication), this time with encryption first.
For SMTP, just to summarize, here is what happens:
– an initial SMTP connection is created unencrypted on port 25
– the client send the command STARTSSL
– a new encrypted connection is created on another port (say, 465 or 587)
Obviously after that I found out that Aruba SMTP server only supports implicit SSL encryption. I think that is because a specific kind of email (called PEC), the only one which has legal validity in Italy, is technically regulated by Italian Law. Aruba is one of the major provider of PEC, and I think they decided to use the same technical standard for all emails. In any case, it is nothing particularly strange: SMTP with implicit SSL encryption. Even GMail uses it, if I don’t mistake, on port 587. Anyway, System.Net.Mail does not support it, so you have to use a deprecated or a third part library.
EVEN MORE IN DEPTH
I think that all of this mess comes from this couple of RFC article: IETF – RFC4217 and IETF – RFC2228. Without going too much into details, they regulates how to use encryption in general and SSL in particular with FTP. Yes, you read right, not SMTP, FTP. From what I understood, alse from other articles, it seems that IETF at the time (somewhere between 1997 and 2005 – about 10 to 20 years ago) favoured EXPLICIT over IMPLICIT SSL connections for two main reasons:
- It simplified the implementation by existent software, not having to rewrite everything but simply adding this functionality in case a specific command was issued;
- It reduced the proliferation of assigned ports in the reserved range (1 – 1024). This was because for every protocol an implicit SSL version required a dedicated port, effectively doubling the number of ports required.
In the meantime, internet happened. There was a proliferation of protocols, implementations (public and proprietary), and of security issues. And then, Microsoft decided “by design” (if you believe that) that they should follow an IETF specification regarding the FTP protocol for their SMTP library. Yes, that’s madness. In fact, the conseguences of this choice is that for the last 10 years this library was useless for half of the SMTPS servers available on the internet, and that a deprecated library (with all the problems it involves) is still widely.
Well done, Microsoft.