Twitter

Wednesday, January 30, 2013

Lync and Exchange Web Services (EWS) and different DNS Domains- Exchange crawling e.g. for presence

Hi all,
(This is an updated version 2.0: 01.20.2014)

there is always confusion in how Lync is crawling Exchange Web Services (EWS).
Generally it is necessary to understand how DNS must be implemented:
Just remember, identify if you have DNS Split configuration, different internal and external DNS names and what are your SMTP and SIP Domains.


Very often you find in Lync/ Exchange deployments an issue, where the Lync configuration show up with empty:
EWS Internal URL
EWS External URL
and
EWS Information = EWS not deployed

Therefor Exchange Web Service are not connected deployed and several Lync Integration Features are not working, e.g. Presence Information based on your Outlook Calendar.
The feature depending on EWS are:
  • Unified Contact Store
  • High-Resolution Photos
  • Meeting tab
  • Contact Information
  • Presence based on Calendar Information
  • Conversation History
  • Missed Conversations
  • Missed Calls
  • Voice Mail Playback





Exchange Setup DNS:

You need PER SMTP Domain 3 DNS Record, internally (Split DNS) and on the external DNS Server, 2x for Autodiscover and 1x for EWS
autodiscover.domain.name CNAME exchangeserver(CAS)
_autodiscover._tcp.domain.name SRV 0 0 443 exchangeserver(CAS)
ewsurl.domain.name A exchangeserver (CAS)
if you use internally another domain, e.g. your Active Directory domain, sure you can have internally another EWS published, but Autodiscover use by Lync identifies still the xml file via the users SIPDOMAIN. So split DNS is recommended (at least for Autodiscover)


NOTE:
As it's never really proper discussed:
Autodiscover will never use the internalURL and externalURL. in Exchange 2007/2010 you are able defining those parameters, in Exchange 2013 they are even documented in TechNet, but they simply don't exist anymore. You'll receive an error if you specify the URLs.

The correct discovery process is like (OUTLOOK):
  • SCP lookup (only if client is domain joined)
  • HTTPS root domain query
  • HTTPS Autodiscover domain query
  • HTTP redirect method
  • SRV record query
  • Local XML file
  • cached URL in the Outlook profile (new for Outlook 2013).

The correct discovery process is like (LYNC):

  • internal, Autodiscover is identified by DNS entry.
  • external, Autodiscover is identified by DNS entry.

Additionally you need to check:


Autodiscovery Virtual Directory:
Setup the internal and external URL, including HTTPS and Basic Authentication
Set-AutodiscoverVirtualDirectory -Identity 'autodiscover (default Web site)' -ExternalURL 'https://ews.domain.name/autodiscover/autodiscover.xml' -InternalURL 'https://ews.domain.name/autodiscover/autodiscover.xml' -BasicAuthentication $true

Note:
The AutodiscoverVirtualDirectoy URL are supposed for Microsoft's optional use only.

Therefore it is not necessary and not Best-Practise defining them!
If you set the URL's, it will NOT HAVE AN IMPACT. Meaning, if they are defined or not, it will not change the Autodiscover behavior, since they are NOT USED within Exchange.
What is IMPORTANT, is the Authentication, you must set it the Basic Authentication, so the SSL configuration will take part. 
It would be enough is you configure simply:
Set-AutodiscoverVirtualDirectory -Identity 'autodiscover (default Web site)' -BasicAuthentication $true
But:
If you define them, you have a reminder what is configured, more like a comment


Web Services Virtual Directory:
Setup the internal and external URL, including HTTPSand Basic Authentication
Set-WebServicesVirtualDirectory -Identity "SERVER01\EWS(default Web site)" -ExternalUrl https://ews.domain.name/EWS/exchange.asmx -InternalUrl https://ews.domain.name/EWS/exchange.asmx -BasicAuthentication $true

The EWS Services are responsible for the Lync integration, especially for Calendar Information and The Conversation History.
Therefore this is the most essential configuration.

Publishing EWS service via Reverse Proxy:
Autodiscover and EWS service do NOT support FBA (form based authentication).
The client need the XML file straight and without authentication webpage, than access the EWS URL need to be authenticated at the Exchange CAS server. Authentication must be NTLM over HTTPS. (So do not use http, the password would be submitted in clear text). The NTLM authentication is hard-coded in Lync Client.


Lync Setup:
First the good new, there is nothing which we have to consider for Lync Server. The Feature is a Client Integration Feature, therefor we have nothing to configure.
There is only one exception, this is the CWA integration for Exchange OWA.
During setup and integration of CWA features, truly the EWS configuration must meet the requirements identically with the Lync Client Configuration.


One last thing necessary to consider and plan proper are the Certificates:
Since all communication is based on HTTPS and TLS, which includes the encryption. Certificates are used for trans-coding.
What is now complicated is the DNS Setup, SMTP/SIP Domains and the SAN Names in this involved certificates.

Lync in this case is straight forward, you simply have to include all SIP Domains in your SAN.
But however Exchange now requires another possible way:
  • make sure you have configured the CAS Server Certificates including all SAN Names for all SMTP and SIP domains
  • make us of IIS based redirection web pages. If you chose this configuration, it is possible minimizing the required SAN configuration.
But still in both configurations you need to consider your DNS Zone setup.
If possible and I personally prefer DNS Splitting, for internal and external resolving. This makes your deployment more supportable.

Note:
if you consult a customer and you are propose DNS Splitting, make sure you fully validate other Web base services, which depends on DNS names too!!


How Lync discover the EWS service via autodiscover:

As illustrated, it is essential for best user experiences having the Lync SIP Domain identically with the default Exchange EMail Domain. Lync is using the smtp-domain for the autodiscovery process. This is especially important if you are not inside your corporate network (LAN). Lync is never able to use SCP, only DNS A and SRV-Records.

DNS resolution occurs first:

  • have look into the Autodiscover.xml file and using the server name (DNS) provided there
  • using autodiscover.<smtpdomain>
  • using _autodiscover._tcp.<smtpdomain>

  • Access now the Autodiscover.xml file located on the Exchange environment in the following order.

    http://<smtpdomain>/autodiscover/autodiscover.xml
    https
    ://<smtpdomain>/autodiscover/autodiscover.xml
    http://autodiscover.<smtpdomain>/autodiscover/autodiscover.xml
    https://autodiscover.<smtpdomain>/autodiscover/autodiscover.xml
    _autodiscover._tcp.<smtpdomain>




    One more remarks:
    If you didn't deploy EWS correctly from the very beginning, you might encounter other Client issues. Therefore it is recommended you delete the following file:

    %userprofile%\AppData\Local\Microsoft\Outlook\*autodiscover.xml

    This file is ONLY created by Outlook, Lync do not write this file it only uses the web services.

    Troubleshooting:
    You should try and access Autodiscover via web browse using a link provided above. You must be asked for your credential (it requires you are having a Exchange Mailbox). Exchange will than show you this XML:

     <?xml version="1.0" encoding="UTF-8"?>
    -<Autodiscover xmlns="
    http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
      -<Response>
        -<Error Id="2907134699" Time="12:55:55.0540898">
             <ErrorCode>600</ErrorCode>
             <Message>Invalid Request</Message>
             <DebugData/>
         </Error>
       </Response>
    </Autodiscover>


    If you see this message, Autodiscover is reachable and ok.

    Next check access to https://EWSURL/ews/exchange.asmx you will be redirected after login with your credentials to: https://EWSURL/ews/Services.wsdl and another xml document is provided.

    Verify also on the client, where Outlook is installed HKCU\Software\Microsoft\x.0\Outlook\Autodiscover\RedirectServers and if necessary delete those entries. Double check those Keys too: HKCU\Software\Policies\Microsoft\Office\x.0\Outlook\Autodiscover
    On the Exchange CAS Servers, you also should check manually on the EWS and the Default Website, if NTLM is the first choice for authentication and NEGOTIATE the second option. You do so, if you open IISManager and use Windows Authentication/ Providers.

    Also see the cache files in Lync, navigate to C:\Users\<user>\AppData\Local\Microsoft\Office\15.0\Lync\sip_uer@domain.name there is file named: EwsFolder<user>@acp.de.cache this file is not readable, so delete it and let is recreate.

    If nothing should help, resetting the Exchange virtual Directories is the last option:
    refer to her: TechNet ff629372

    Note:
    The Registry Key under HKCU\Software\Microsoft\Office\x.0\Lync\<user.name>\...
    here LyncAutodiscover is not for EWS, it caches the LYNC WEB SERVICES only.
     


    62 comments:

    1. Hello!
      Is it possible, that Lync2013 Client NEVER uses the
      _autodiscover._tcp.domain.name
      but always assumes to be able reaching https://autodiscover.domain.com
      instead?

      ReplyDelete
    2. Hi Harald,
      with autodiscover we have four different scenarios.
      1. outlook client, it first queries the smptdomain.com/autodiscover, than https/http autodiscover.domain and at last the SRV record
      2. Lync Client, its similar to outlook, therefor the SRV record will be queried last
      3. Lync Server, it queries only the A/CNAME records as you have defined it also for other trusted services.
      4. Exchange Server, it queries in the same ranking as Outlook, but it depends on Outlook Anywhere is enabled or not.

      I hope this helps you a little bit, generally, Best Practice is having both A and SRV record.

      ReplyDelete
    3. Hi Thomas,
      I'm using Exchange 2013, and I try to set the autodiscover virtual directory external URL as you suggest in the Exchange Power Shell, but the command to set it in Exchange 2013 apparently doesn't exist.
      I verify that your commands are correct for exchange 2010 but not for exchange 2013. Do you have any idea to configure it in Exchange 2013.
      Thanks

      ReplyDelete
    4. Hi Jose,

      this is a very good comment. Well you are right it is working in Ex2k7, Ex2k10 but NOT in Ex2k13.
      Why is this so..?
      The Autodiscover URL was never used in Exchange, in non of the existing versions!

      The Internal Process is via SCP and the External Process is DNS only.
      True there are a lot of information not optimal in TechNet right now. Even this is a normal mistake.

      If you run the autodiscover test in Outlook, you'll see , also this on in TechNet, what will be queried and when.

      I have added one sentence about this now in the Blog.
      Thanks

      ReplyDelete
    5. What if you have multiple cas array's in your organization? Where do you point the "autodiscover.domain.name CNAME exchangeserver" to?

      ReplyDelete
    6. Hi "Anonymous",
      well its still an Lync Blog, but regarding your question where to point the autodiscover if you have multiple CAS Arrays.

      Exchange 2007/ 2010:
      You point internally to most used CAS Array, in case of this Array dies, you reassign to another one. Externally seen its the same, you point to the main published site.
      If you have GEO DNS, you can point to the nearest.

      Exchange 2013:
      There is no such thing as CAS Array, you only have load balances CAS Services. Else 2013 recommends GEO Data based HA, therefor its similar to the older version, point autodiscover to the largest site and repoint if this site fails.

      Regarding the question if it can be a CNAME, well it can be. just point it the DNS name of your CAS Array or the load balance shared IP

      I hope it helps

      ReplyDelete
    7. Hi Thomas,
      I am facing similar issue and did the change as per your article with little change "https://autodiscover.domain.com/autodiscover/autodiscover.xml" in autodiscover virtual directory. But I am bit confused on DNS entry, should I make entry like "dnscmd . /zoneadd _autodiscover._tcp.domain.com. /dsprimary" and then "dnscmd . /recordadd _autodiscover._tcp.domain.com. @ SRV 0 0 443 mail.domain.com" ?

      I have CAS array and in DNS there is one pinpoint zone "mail.domain.com" with A record to CAS Array IP. One more zone "autodiscover.domain.com" with same A record to CAS Array.

      In CAS Server, mail.domain.com redirect to mail.domain.com/owa. May be this is the reason EWS not resolving in Lync 2010 client. Plus IPhone error "cannot connect to exchange web server".

      My internal domain is domain.local and external is domain.com (no entry of domain.com in internal DNS).

      Please help. Thanks

      Prabodha

      ReplyDelete
      Replies
      1. Hi Prabodha,

        you only need the A record pointing to mail.domaincom (CAS Array) in your case and a SRV record also pointing to mail.domain.com

        Delete
      2. Thank you Thomas for the reply. But still not resolving. I just discussed with IT Head and removed all pinpointed zones. The primary zone is "domain.local". I added "domain.com" in internal DNS and add all 'A' records for :

        (Exchange 2010 SP1 - Godaddy SSL Certificate on CAS servers and TMG published)
        mail.domain.com - CAS Array IP (outlook auto resolving to this URL)
        autodiscover.domain.com - CAS array IP
        lyncdiscoverinternal.domain.com - Lync FE IP
        meet, dailin, sip - Lync FE IP

        www.domain.com - public hosted IP for internal user access website.

        SRV record - _autodiscover._tcp.domain.com SRV 0 0 443 mail.domain.com
        and _sipinternaltls._tcp.domain.com SRV 0 0 5061 sip.domain.com

        CAS autodiscover - https://autodiscover.domain.com/autodiscover/autodiscover.xml

        EWS internal & external are same - https://mail.domain.com/EWS/Exchange.asmx

        Lync 2010 standard Front End - reapply SSL Certificate from internal CA (DC). Lync Edge Internal SSL from internal CA (sip.domain.com) and external from Godaddy SSL Certificate with SAN included (Dialin, meet, lyncdiscover, lyncdiscoverinternal, sip.domain.com) and published on TMG with another listener.

        Everything looks fine, but still the issue persist. My doubt going towards authentication. From outside when Outlook Anywhere login pop-up does not resolve username@domain.com but resolve to username@domain.local or domain\username. Is this causing Lync client not able to get EWS?

        Please help, I tried to get help in a lot of forum but no help. MSFT team told me to upgrade to exchange 2010 SP2 (but my doubt if EWS URL resolving from IE explorer, then why not Lync client)

        Thanks & Regards,

        Prabodha

        Delete
      3. Hi,
        I can see some ???
        The Edge Internal Interface only need a Certificate with the FQDN (internal) of this server.
        The external certificate do not have the Web Service Name.

        The Exchange user name resolver issue is properly in you AD, which use the AD suffix you have given. if you wanna change this, you need to change the logon suffix in AD.
        IE has nothing to do with the client/ server beside the Proxy settings, so you need to validat, the lync stays internal if it queries via a proxy!

        Delete
    8. Replies
      1. Dear Thomas, ever since i upgraded my Lync 2010 to 2013 I have on my front end a repeating error (every thirty minutes):Storage Service had an EWS Autodiscovery failure
        And for lync client it will always ask for exchange credentials without any sign in success
        Please Advise

        Delete
      2. The second issue you can solve if you recreate the Outlook Client Profile.
        This issue should not appear on each client, well?

        Let me clarify, you mean the LS Storage Service well.
        This could lead to two different problem.
        1. Your Autodiscovery Setup isn't correct.
        2. You have a Proxy Config active on your Lync Server. Change this to "no proxy" than it should be fine.

        Delete
    9. What command did you use to get the result in the first picture ?

      ReplyDelete
      Replies
      1. This is the Lync client:

        you need to click the "hidden icons" lower right side of desktop, than "right click" the LYNC symbol and chose: "Configuration Information"

        Than this info screen will pop-up

        Delete
    10. Thank you Thomas for your reply.
      Sorry for my late response, can you show me how change my FE to no Proxy?
      This is Elie Hajj

      ReplyDelete
      Replies
      1. Hi Elie,
        you can follow the instruction in my other post:
        http://lyncuc.blogspot.de/2012/12/lync-exchange-certificates-crl-check.html

        this is not only for the CRL Check, it also shows you how to use: netsh winhttp

        Delete
      2. Thank you for your response Thomas but it seems that it's not my case.
        My issue is the same as this post

        http://social.technet.microsoft.com/Forums/en-US/lyncdeploy/thread/c5bf2775-d195-4f3a-944d-733d707ab698?Thread%3Ac5bf2775-d195-4f3a-944d-733d707ab698=Microsoft.Forums.Data.Models.Discussion&ThreadViewModel%3Ac5bf2775-d195-4f3a-944d-733d707ab698=Microsoft.Forums.CachedViewModels.ThreadViewModel

        Which is still till date not resolved.
        If you can help, will appreciate it

        Thank you
        Elie Hajj

        Delete
    11. Hi,
      We have same kind of error and does not seem to figure out how to fix it.
      Lync2013 Client EWS does not work on external network via TMG.
      But in internal network it works directly to Exchange and connects to ews external web uri not the internal...

      In TMG we have published autodiscover and ews with basic auth nothing else no exchange anywhere.

      It seems that with browser after auth popup you get the ews 'xml', but when I debug with Fiddler I see that Lync2013Client gets autodiscover OK and sends POST to ews-url and gets 401 Unauthorised from TMG but does not respond with credentials to TMG's 401 Unauthorised.
      Any toughts ?

      ReplyDelete
      Replies
      1. Tomi, have you solved this? We have the exact same issue. Basically, our ews directory is still under the mail.domain.com url, so even though we have published the autodiscover and ews folders with a separate listener and publishing rule, the ews folder is being requested from the mail.domain.com rule and thus hitting the Form based authentication rule...?
        Thanks

        Delete
    12. Hi Tomi,

      this is all Exchange related. You are not allowed to authenticate the client on the TMG. There are some more nice blogs about Exchange, e.g. msexchange.org. you can also check with www.testexchangeconnectivity.com
      Set the option to: "No delegation, but client may authenticate directly"
      Also helpful TechNet: http://technet.microsoft.com/en-us/library/bb124251.aspx
      And remember, beside OWA, no Form Based Authentication is allowed

      ReplyDelete
    13. Hi Thomas ,

      i am deploying F5 hardware load balancer for load balancing two Exchange 2013 CAS servers .

      outlook is working fine from client side , but when i am opening Lync 2013 client i get username/password prompt and never get authenticated .

      i am doing decryption/encryption on F5 load balancer [client side SSL certificate and server side SSL certificate] used to receive and decrypt the client packet , read it then encrypt it back to the CAS server ] .

      i am testing using local host file on client machine , so will this work or i need an A record and SRV record on DNS ?

      thanks ,
      Yazan Khader

      ReplyDelete
    14. Hi Yazan,

      you are no having an EWS issue, even if it looks like.
      You problem is the Outlook Profile. You have two choise solving this issue by:
      a) repair your Outlook Profile
      or
      b) create a new profile for Outlook,

      than this problem is shall diappear.
      Thomas

      ReplyDelete
    15. In regards to the EWS Virtual Directory permissions why BasicAuthentication? Could you have used DigestAuthentication or Windows Authentication?

      Basic seems to open and Windows may cause issues with authentication when you can't reach a DC.

      ReplyDelete
    16. Hi Les Chafin,

      this is quite logic due to TLS encryption. You need to have a valid certificate deployed as usual with autodiscover.
      If you are connecting from internal LAN, the other authentication methods are not disabled, which means, Windows Integrated authentication on the internal URL are still active. This must also be like this, because internal atuodiscover will also be provide by SCP (Service Connection Point) with is defined in AD under the CAS Server.

      In you second paragraph, you mentioned about issues with Basic authentication if the DC is not available. True, if a DC is nto available, simply no authentication will work, neither basic nor windows integrated.

      Back to the Basic Authentication, if this is not active, any external authentication will fail because you need access via https and this require (also depending on your firewall deployment) basic authentication.

      hope this helps for some better understanding.

      ReplyDelete
    17. Hello Thomas, I have some problem with Lync 2013 EWS integration
      I have domain : domain.local
      mail addresses: user@domain.com
      I have some users that must use user@domain1.com for external mails
      And i have a problem with those people, their Lync Clients don't have any EWS addresses in their properties.
      Can you help me with this problem

      ReplyDelete
      Replies
      1. Better late than never:

        Well this is a common missunderstand in how autodiscover works.

        REMEMBER:
        Autodiscover is ALWAYS Associated with the users primary email domain, in our case even more Exchange and Lync have the same domain in common. this requires a consitent deployment for each SIP and users primary email domain, especially internally in conjunction with SPLIT DNS.

        this is concept is brocken, you will have isuess.

        hope it helps you solving your case

        Delete
    18. Lync 2013 EWS is noot Deployed in Our Desktop

      ReplyDelete
      Replies
      1. Hi Abdul,
        you need than to make sure your autodiscover is setup with exchange according to the deployment guide

        Delete
    19. Many thanks, ISSUE SOLVED on ARR 2.5 IIS 8 + EXCHANGE 2013 + LYNC 2013 + multi tenants + one SSL domain with HTTP redirect for subdomains

      Lync 2013 EWS will not work with HTTP redirect on autodiscover, neither with SRV record, you need the A record + HTTPS autodiscover

      ReplyDelete
      Replies
      1. Hi Thomas,

        Do you mind elaborating how you made this work? What do you mean by "you need the A record + HTTPS autodiscover"?

        Delete
      2. I have a similar configuration as yours but I'm using TMG for the reverse proxy. I got this working last night by using SRV record for the redirection with one SSL domain for a multitenant setup.

        Delete
      3. Hi Conrado, good you sort this out.

        Delete
    20. I cannot do Set-AutodiscoverVirtualDirectory because I don't have a ClientAccessArray (we have only 1 exchange server). SRV records exist and Set-WebServicesVirtualDirectory is correctly set.
      autodiscovery works for Outlook but Lync says 'EWS not Deployed'

      Do I need to create a ClientAccessArray? What is th eimpact if I do that?

      Rob

      ReplyDelete
      Replies
      1. Hi Rob,
        this is a Exchange related feature CAS Array and do only exits up to Exchange 2010, E2013 doesn't have this anymore. Therefore if you only a single server, the CAS Server is part of this consolidated setup and has the AutoDiscover entry as well. So go ahead with the described setup as in this blog.
        Tnx
        Thomas

        Delete
    21. that's the problem, I can't go on...
      If I do this: Set-AutodiscoverVirtualDirectory -Identity 'autodiscover (default Web site)' -ExternalURL 'https://mail.domain.name/autodiscover/autodiscover.xml' -InternalURL 'https://mail.domain.name/autodiscover/autodiscover.xml' -BasicAuthentication $true

      Exchange 2013 says:
      Cannot process argument transformation on parameter 'InternalUrl'. Cannot convert value "https://is-exch-002.office.is.
      nl /autodiscover/autodiscover.xml" to type "System.Uri". Error: "Invalid URI: The hostname could not be parsed."
      + CategoryInfo : InvalidData: (:) [Set-AutodiscoverVirtualDirectory], ParameterBindin...mationException
      + FullyQualifiedErrorId : ParameterArgumentTransformationError,Set-AutodiscoverVirtualDirectory

      according to this it does exist in E2013: http://technet.microsoft.com/en-us/library/dd297976(v=exchg.150).aspx

      ReplyDelete
      Replies
      1. Hi Rob,
        just try to explain this:

        The AutodiscoverVirtualDirectory (InternURL/ExternalURL) are optional parameters and as Microsoft said for internal MSFT use.

        What does this mean:
        The AutodiscoveryVirtualDirectory is not need!

        Why !?!
        This is due to, if the autodiscover is set correctly in AD SCP and also DNS. It is not necessary using this parameters.

        But:
        Mostly there are so many things wrongly configured in Exchange, that I spoke with some Exchange Gurus, that it is possible using this parameter and make you life more easy.

        Summary:
        If AD and DNS is set correctly, EWS vDir and also the authentication method is sufficient enough.

        Back to your problem:
        I believe you have a typing mistake, since the parameter will work. Check your typing. I think I saw a blank in the name.

        Delete
    22. Hi Thomas,

      We currently have Exchange 2007/Lync 2013 deployed. Exchange has EWS deployed, however, the Internal/External Autodiscover URLs are not configured. As you've pointed out, these URLs are pointless if you have your DNS records in place. Now, onto the problem:

      EWS fails in the Lync client. I cannot see the Meeting icon in Lync and my recent conversations are not listed. However, Lync does change my presence status based on if I have a meeting or not schedule in Outlook. I've confirmed that all of the DNS records are in place and that the WebServices directories are created and functioning. Yet, this still fails.

      Any ideas?

      ReplyDelete
      Replies
      1. Hi Courtney,
        just first, I updated the article today. Making things more clear.
        If you have the problem internally, delete the described xml file.
        Other related problems, maybe with you certificate or load balancers.
        Can you check this please?

        Delete
      2. Here's what I did:

        1) I checked to make sure that basic authentication was set to 'true' for the URLs and for the Web Services piece. All of that was already set.

        2) I've turned up logging on the Lync client. Let me know if you'd like to see the logs so we can maybe figure out where things are failing.

        3) Which certificates do I need to check for Lync and Exchange? I'm not very certificate savvy so any info you can provide would be much appreciated.

        4) Autodiscover is working in every other way possible. I was able to use the 'Test E-Mail Autoconfiguration' and Outlook pulls all of the EWS info correctly.

        Delete
      3. I also logged in with a Lync 2010 client and checked it's configuration information. It also shows EWS is not deployed and that Exchange connectivity is down. The 2010 client did prompt me with a "Lync is attempting to connect to autodiscover.domain.com. Certificate Details: mail.domain.com. Lync cannot verify that the server is trusted for your sign-in address. Connect anyway?"

        I selected "connect", but the EWS still failed to pull through. I've also noticed that I'm missing entries in the registry for the Lync 2013 client. Under HKCU\Software\Microsoft\Office\15.0\Lync\sip@domain.com\LyncAutodiscovery there should be strings that indicate what the EWS sites are. This info, however, is missing. This tells me that the Lync 2013 and 2010 client are completely in capable of pulling this info.

        Delete
      4. Hi Courtney,
        this was a very good finding. certificate assignment is very crucial. Than you problem 1. is simple. Make sure the autodiscover SAN entry is set in the Exchange Cert.

        Delete
      5. We do have the autodiscover.domain.com entry in the Exchange cert. However, it's not the same domain that my Lync servers are joined to. So, we need to add autodiscover.sipdomain.com into the Exchange cert to fix this, correct?

        Delete
      6. You made a good point.
        there are dependencies with the DNS names. Truly in Exchange and DNS you need to define autodiscover, therefore in the certificates too.
        Just to explain how Lync is use autodiscover for EWS services:
        http://smtpdomain/autodiscover/autodiscover.xml
        https://smtpdomain/autodiscover/autodiscover.xml
        http://autodiscover.smtpdomain/autodiscover/autodiscover.xml
        https://autodiscover.smtpdomain/autodiscover/autodiscover.xml
        _autodiscover._tcp.smtpdomain

        So your SIP Domain hast be the same as your EWS Domain, else the Lync client cannot search for Exchange.

        On a desktop client its different, Lync can use 1. Outlook and also query the SCP in AD. So suspect internally your environment is working.

        Delete
      7. Well, I wish that were the case, but unfortunately this is broken internally as well. When Lync tries to hit autodiscover, it searches for autodiscover.domain.com instead of autodiscover.sipdomain.com. I'm thinking that once we add the autodiscover.sipdomain.com to the cert, things will be perfect.

        Delete
    23. Hi Thomas
      I have a really strange issue and I'm wondering if you have some suggestion before opening a MS support request.
      I have a Lync2013+Exchange2010 cluster perfectly working.
      smtp domain for all users is smtpdomain.eu and sipdomain for all users is sipdomain.com.
      Autodiscover and certificates deployed correctly.
      For users on Windows XP+Lync2013 (Basic)+Outlook2010 everything works perfectly (autodiscover, exchange EWS and MAPI integration...)
      Users on Windows 7+Lync2013 (Basic)+Outlook2010 everything works but EWS.

      If I configure user with sipuri domain==smtpdomain, it works
      If I use fiddler with https decryption it works!
      If I use the same users on another PC (XP) it works...

      I think there should be an issue with authentication that involves the sipdomain of the user, but I see no forbidden errors on CAS IIS... Any suggestion on what could I check?

      Final note: both windows xp machines and windows 7 machines are deployed from an image.

      Many thanks!

      ReplyDelete
      Replies
      1. Hi Mat, if you are inside your LAN, there is another method for autodiscover. The so called SCP.
        It seems here is a mismatch between you SCP and the DNS Domain Split.
        can you verify this and come back with your findings?

        Delete
      2. Hi Thomas.
        As pointed out by some other users, Lync client doesn't use SCP to find Exchange autodiscover.
        Anyway after many tries (rebuild autodiscover and EWS Virtual Directories, change EWS and autodiscover authentication mechanism, check certificates chains and CRL, root certificates on servers and clients and so on...) I found this solution:
        http://uclobby.com/2014/01/03/lync-client-2013-ews-has-not-fully-initialized/
        I though it was a certificate related issue, since with fiddler in the middle it worked and what I saw in network monitor was a first connection to autodiscover url then nothing else.

        What was strange to me was that exchange server hostname is listed in SAN certificate, and had no popup error when connecting... But it seems that in latest Lync 2013 client releases that popup is suppressed...

        So, adding that key (by hand or by GPO) adding SMTP domain as trusted by Lync solved the issue, bypassing SMTP domain verification.

        Delete
      3. Hi Mat, you are welcome, I appreciate all kind of questions and even more, if you share information helpful to others too.
        Thank you very much.

        What David is mentioning here is a problem, where you have not set the AD Suffix equal to you email/ SIP domain. There is a protection mechanism, which also has this impact he described. I personally haven't see this issue, but will verify this soon and probably update this article.

        I explained some more information regarding the untrusted certificate from a Lync client here: http://lyncuc.blogspot.de/2013/06/lync-client-certificate-authentication.html

        Delete
      4. Thank to you for your full-of-detailed-information blog ;)
        what was strange in my case is that UPN suffix WAS the same as the SIP address (change while implementing Lync to make logins smoother :D )...
        It remains a wonder why it worked with Lync 2010 clients or on XP workstations (with no errors), and what was very disappointing is that we didn't receive warnings "Lync cannot verify that the server is trusted for your sign-in address. Connect anyway?" (that both you and David quoted).

        Delete
    24. hi,

      I tried every possible solution over the internet but I'm not able to solve the problem of EWS not deployed in my environment.

      I have a Host A record defined with name and an SRV record.
      The Lync Client resolves this.

      Name: autodiscover.testlab.local
      Address: 192.168.10.25

      an SRV record
      _autodiscover._tcp.testlab.local SRV service location:
      priority = 0
      weight = 0
      port = 443
      svr hostname = exg.testlab.local
      exg.testlab.local internet address = 192.168.10.25


      I also tried accessing the URL https://exg.testlab.local/EWS/Exchange.asmx
      from internet explorer and it opens (but with certificate error since it is an internally generated certificate)

      can you guide me what am I missing here ??

      Waiting for your response Thanks..

      ReplyDelete
      Replies
      1. Hi Muhammmad,
        did dint not fully catch your problem, are talking about Internet, do you?
        If so, two things are important, sure the DNS setting in the real world.
        And, and this looks like your issue:
        The correct certificate deployment.
        If you assigned a private certificate, e.g. from your own internal CA. You need to ensure the Root Certificate is trusted from the Client you us outside (Internet).
        It will not work, if the certificate issue is still present.
        Just export the Trusted Root Certificate and import it into you test client.
        than it should work

        Delete
      2. Hi,

        thomas... The whole setup is internal ... no internet involved. I meant to say that I tried every possible solution by searching the internet and different blogs ...

        Yes the CA is internal ... I hope you get the whole story now.

        Looking forward to your response . thanks .

        Delete
      3. Hi Muhammad, as you wrote and answered, the problem is you CA deployment. If you have nor trusted root certificate in the local computer store, nothing will work, neither EWS nor a trusted IE web access as you tested.
        beside this, in the certificate, the local server fqdn and also autodiscover fqdn must be in SAN entries.
        that's you issue.
        the rest should not be any problem, if you followed the advice from this blog setting up a EWS with Lync

        cheers and I hope you can easily solve this matter

        Delete
      4. Hi Muhammad,
        please read the comment I wrote today, it just gives you another hint and explanation.

        Delete
    25. This is especially important if you are not inside your corporate network (LAN). "Here Lync is able to use SCP" - I don't think this is correct.

      Lync clients will only use DNS routing even in internal and not SCP is what I believe

      ReplyDelete
      Replies
      1. Hi SP,
        thank you so much.
        Yes you are right and found the typo.
        Lync is not able querying SCP in AD, only DNS.
        In DNS Lync Client queries the A-Record autodiscover.domain.com

        There is currently a BUG in Lync Client 2013:
        Can't Autodiscover Exchange Server from DNS SRV records

        Lync Server can't automatically discover an Exchange Server using DNS SRV records. Instead Lync Server must use the HTTP Autodiscovery feature to find the Exchange Server.

        Workaround Ensure the HTTP Autodiscovery feature is properly deployed in your Exchange Server environment so Lync Server can connect to the Exchange Web Services.

        http://office.microsoft.com/en-us/lync-help/lync-2013-known-issues-HA102919641.aspx

        I don't want putting this bug into the EWS blog, since I hope this could be solved by Microsoft.

        Delete
    26. Hello everybody,

      Is there a possibility to prevent the Lync client trying to resolve the autodiscover files from the blank sip domain?
      http://smtpdomain/autodiscover/autodiscover.xml
      https://smtpdomain/autodiscover/autodiscover.xml

      In Outlook there is the "ExcludeHttpsRootDomain" registry key. Is there something similar for the Lync client? I couldn't find any documentation about this so far.

      Thanks in advance!

      ReplyDelete
      Replies
      1. Hi
        this is as far as I know not possible, because the Autodiscover process is fixed in the client and the Autodiscover registry keys are in profile:

        Registry Key For Lync 2010 :
        HKCU\Software\Microsoft\Communicator\\Autodiscovery

        Registry Key for Lync 2013:
        HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Lync\<\Autodiscovery

        Delete
    27. I all, I updated this article today with more relevant information based on Exchange Server, Lync Server and the Outlook Client, as well as the Lync Client.

      Hopefully now it answers all questions.

      Thanks to all for your great contribution and interest in this article.
      Thomas

      ReplyDelete
    28. Hi, Just some additional information. EWS was working OK on our internal Lync 2010 clients, it stopped working on our Lync 2013 clients (also internal). It turns out that the 2013 client uses the proxy settings of the system, so if you have a proxy configured on the client PC and the autodiscover.sipdomain is not excluded from going through the proxy then the Lync client won't resolve autodiscover.sipdomain but just sends a get request for autodiscover.sipdomain/autodiscover/autodiscover.xml to the proxy. This failed in our case as the proxy only connects to the outside. The Lync client then tries to resolve the SRV record, it skips the A record. We did not have the SRV record, so our EWS connection failed.

      So be aware that if your SIP domain is normally handled by your proxy that you either create the SRV record or add autodiscover.sipdomain to the proxy exclusion list

      ReplyDelete
      Replies
      1. Hi Pete,
        you are absolutely right!
        For testing those settings I explained a manual change you can still use for clients too.
        http://lyncuc.blogspot.de/2012/12/lync-exchange-certificates-crl-check.html

        It is hopefully well know, if your internal clients use a proxy server, the IE settings are valid for all Microsoft applications. Therefore you need to be aware of your internal setup and the path the application will go.

        tnx

        Delete