Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WiX 3.9 R2 lit.exe/light.exe throw IOExceptions Ending the Build When MultiProc is Enabled under TFS 2103 Team Build #4791

Closed
wixbot opened this issue Jun 9, 2015 · 7 comments · Fixed by wixtoolset/wix#258
Assignees
Milestone

Comments

@wixbot
Copy link

wixbot commented Jun 9, 2015

We currently have an open support ticket with Microsoft for significantly increasing build times since shortly after the migration to TFS 2013 in January 2015. We are under significant pressure to re-enable MultiProc (on by default) on most of our builds. Two issues result when building under MultiProc:

  1. lit.exe/light.exe fail with an IOException. This affects the building and incorporation into our installers of: 1) our DACPAC's from sqlproj; 2) large assemblies/executables; 3) ZIP modules used by our admin consoles; and 4) *.config files that are processed by XSLT transforms during the build process.

WORKAROUND: Build with MultiProc disabled (single threaded builds). Cost is a 10-15 minute increase in build times.

  1. On one of our builds with an especially large ZIP module has experience random truncation of the ZIP module incorporated into the installer. Research suggests that a deadlock bug in Dot.NET Zip causes it to sometimes stall outputting blocks to the ZIP. It appears that during this stall, the ZIP module is sometimes incorporated into the MSI. The corruption removes some or all of the directory entries from the ZIP--rendering it useless.

WORKAROUND: Disable MultiProc with a commensurate (for this build) 5-7 minute increase in build times.

As this issue is rather important to the company, my request for approval to enter into an assignment agreement has been elevated to the CTO who has deposited the agreement for review on the chief counsel's desk. I am supposed to know by the end of the week.

Meanwhile, I am working on modifications to lit.exe/light.exe to mitigate this issue, and I have made some progress. So, I will provisionally take it.

Originally opened by jmcooper8654

@wixbot
Copy link
Author

wixbot commented Jun 10, 2015

Well, the stack frame is not well preserved. It's telling me that my exception is coming out of the Run() method of Lit which is a little underwhelming. That leaves me with implementing tracing, adding throws to preserve the stack frames, or adding stacktrace blocks to the deeper catch blocks. I'm going with tracing first.

Originally posted by jmcooper8654

@wixbot
Copy link
Author

wixbot commented Jun 11, 2015

What the trace log looks like so far:

[2015-06-11 08:33:55Z] LGHT(I): Entering light.
[2015-06-11 08:33:55Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsIn
stallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 08:33:55Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsIn
stallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 08:34:00Z] LGHT(I): Leaving light.
[2015-06-11 08:34:14Z] LIT(I): Entering lit.
[2015-06-11 08:34:19Z] LIT(I): Leaving lit.
[2015-06-11 08:34:20Z] LGHT(I): Entering light.
[2015-06-11 08:34:23Z] LGHT(I): Leaving light.
[2015-06-11 08:35:53Z] LIT(I): Entering lit.
[2015-06-11 08:35:59Z] LIT003(E): The process cannot access the file because it
is being used by another process. (Exception from HRESULT: 0x80070020)
[2015-06-11 08:35:59Z] LIT(I): Leaving lit.

LIT003 is the last, generic catch in the LIt.Run() method (I'll need a better naming convention for the trace--but it was quick). The HRESULT is clearly for a sharing violation. Although frustrating, I continue to dig to illuminate this further.

Originally posted by jmcooper8654

@wixbot
Copy link
Author

wixbot commented Jun 11, 2015

Progress. First MultiProc build of a 10 solution, 2 configuration product:

[2015-06-11 13:17:45Z] LGHT(I): Entering light.
[2015-06-11 13:17:46Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:17:46Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:17:51Z] LGHT(I): Leaving light.
[2015-06-11 13:18:04Z] LIT(I): Entering lit.
[2015-06-11 13:18:09Z] LIT(I): Leaving lit.
[2015-06-11 13:18:10Z] LGHT(I): Entering light.
[2015-06-11 13:18:13Z] LGHT(I): Leaving light.
[2015-06-11 13:19:41Z] LIT(I): Entering lit.
[2015-06-11 13:19:47Z] LIT(I): Leaving lit.
[2015-06-11 13:20:06Z] LIT(I): Entering lit.
[2015-06-11 13:20:13Z] LIT(I): Leaving lit.
[2015-06-11 13:20:41Z] LIT(I): Entering lit.
[2015-06-11 13:20:47Z] LIT(I): Leaving lit.
[2015-06-11 13:21:37Z] LIT(I): Entering lit.
[2015-06-11 13:21:44Z] LIT(I): Leaving lit.
[2015-06-11 13:23:45Z] LIT(I): Entering lit.
[2015-06-11 13:23:51Z] LIT(I): Leaving lit.
[2015-06-11 13:23:55Z] LGHT(I): Entering light.
[2015-06-11 13:23:55Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:55Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:55Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:56Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:56Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:56Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:56Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:23:56Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:24:34Z] LGHT(I): Leaving light.
[2015-06-11 13:24:39Z] LGHT(I): Entering light.
[2015-06-11 13:24:51Z] LGHT(I): Leaving light.
[2015-06-11 13:25:48Z] LGHT(I): Entering light.
[2015-06-11 13:25:49Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:25:49Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:25:54Z] LGHT(I): Leaving light.
[2015-06-11 13:26:07Z] LIT(I): Entering lit.
[2015-06-11 13:26:13Z] LIT(I): Leaving lit.
[2015-06-11 13:26:14Z] LGHT(I): Entering light.
[2015-06-11 13:26:16Z] LGHT(I): Leaving light.
[2015-06-11 13:27:03Z] LIT(I): Entering lit.
[2015-06-11 13:27:09Z] LIT(I): Leaving lit.
[2015-06-11 13:27:28Z] LIT(I): Entering lit.
[2015-06-11 13:27:35Z] LIT(I): Leaving lit.
[2015-06-11 13:28:02Z] LIT(I): Entering lit.
[2015-06-11 13:28:08Z] LIT(I): Leaving lit.
[2015-06-11 13:28:56Z] LIT(I): Entering lit.
[2015-06-11 13:29:03Z] LIT(I): Leaving lit.
[2015-06-11 13:30:03Z] LIT(I): Entering lit.
[2015-06-11 13:30:09Z] LIT(I): Leaving lit.
[2015-06-11 13:30:14Z] LGHT(I): Entering light.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:14Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:15Z] LGHT001(W): Exception of type 'Microsoft.Tools.WindowsInstallerXml.WixNotIntermediateException' was thrown.
[2015-06-11 13:30:56Z] LGHT(I): Leaving light.
[2015-06-11 13:31:00Z] LGHT(I): Entering light.
[2015-06-11 13:31:12Z] LGHT(I): Leaving light.

This build was taking between 40 and 60+ minutes build. This build was 17.3 minutes.

Need to stress test to make sure the results are permanent.

Originally posted by jmcooper8654

@wixbot
Copy link
Author

wixbot commented Jun 16, 2015

Checked against both build with IOException sharing violation issue and build with truncated ZIP module. Modifications to CompareFiles() and LayoutMedia() appear to work. Senior manager has directed me to put the fix into production on all four build servers--I will be doing that tonight. Side benefit: modifications to CompareFiles() and allowing MultiProc to work cut a 50 minute build down to 17-19 minutes. Putting up pull request presently.

Originally posted by jmcooper8654

@wixbot
Copy link
Author

wixbot commented Jun 30, 2015

Release changed from v3.9 to v3.11

@wixbot wixbot added this to the v3.11 milestone Dec 20, 2015
@barnson barnson self-assigned this Feb 14, 2017
@barnson barnson added the triage label Feb 27, 2017
@barnson
Copy link
Member

barnson commented Feb 27, 2017

John's full PR is too big for wix3. I tried to implement a subset with retries but ran into the need for retries in many different spots.

@barnson barnson modified the milestones: v4.0, v3.11 Feb 28, 2017
@barnson barnson removed the triage label Mar 28, 2017
@rseanhall rseanhall added this to To do in wix.4.0-preview.1 Jun 14, 2021
@barnson
Copy link
Member

barnson commented Jul 21, 2022

Investigate to ensure all the i/o goes through the file system object with retries.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

4 participants