samtools incompatible library version

[yaaminiv]

Trying to merge sorted bam files with samtools merge in this Jupyter notebook like @kubu4 suggested in this discussion. I ran the following code:

!/Users/Shared/Apps/samtools-1.8/samtools merge \
-ur \
-b /Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_1_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_2_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_3_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_4_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_5_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_6_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_7_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_8_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_9_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_10_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_11_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_12_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_13_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_14_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_15_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_16_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_17_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_18_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_19_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_20_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_21_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_22_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_23_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
/Volumes/web/spartina/project-oyster-oa/Haws/bismark-2/zr3644_24_R1_val_1_val_1_val_1_bismark_bt2_pe.deduplicated.sorted.bam \
merged-sorted-deduplicated.bam

and got this error:

dyld: Library not loaded: /usr/local/opt/xz/lib/liblzma.5.dylib
  Referenced from: /Users/Shared/Apps/samtools-1.8/samtools
  Reason: Incompatible library version: samtools requires version 8.0.0 or later, but liblzma.5.dylib provides version 6.0.0

What is this library, and how do I update it?

[kubu4]

I'd upgrade to a newer version. I think that will be easiest.

http://www.htslib.org/download/

[yaaminiv]

I was able to upgrade to htslib version 1.12! I'm running into an error downloading the latest samtools version. I downloaded the latest version on genefish and ran ./configure --prefix=/Users/Shared/Apps/samtools-1.12. Got the following error:

config.status: creating config.mk
dyld: Library not loaded: /usr/local/opt/mpfr/lib/libmpfr.4.dylib
  Referenced from: /usr/local/bin/gawk
  Reason: image not found
./config.status: line 879: 83237 Done(141)               eval sed \"\$ac_sed_extra\" "$ac_file_inputs"
     83238 Abort trap: 6           | $AWK -f "$ac_tmp/subs.awk" > $ac_tmp/out
config.status: error: could not create config.mk

Without the config file I can't finish installing the program. I assume I need to find the dyld library/get a new one?

[kubu4]

Is homebrew installed? If yes, try upgrading gawk first:

brew upgrade gawk

I won't be able to help very much, as I'm unable to SSH into genefish (I'm using the IP address in the Roberts Lab Computer log - is that still correct?). Along those lines, any reason you're not using one of the other birds? They're all faster and have more memory than genefish.

[yaaminiv]

try upgrading gawk first

I do have homebrew! Just ran brew upgrade gawk and got the following error:

I'm unable to SSH into genefish (I'm using the IP address in the Roberts Lab Computer log - is that still correct?)

That's the IP address I'm using!

Along those lines, any reason you're not using one of the other birds? They're all faster and have more memory than genefish.

I'm using genefish since I can Screen Share into it from my Mac. Lazy, but it works for running Jupyter notebooks on something that isn't my own computer (or...at least it usually works without too many hiccups)

[kubu4]

So, that seems like one possible issue. Install gawk and try the samtools configure stuff again:

brew install gawk

[yaaminiv]

Okay, re-installed homebrew, installed gawk (may have run rm '/usr/local/bin/awk' to try and link gawk based on installation prompts without thinking.....that's a problem for later), and I'm still getting the same error

edit: awk is still on my machine I did not totally screw up

[yaaminiv]

Okay, did some digging in the Homebrew Github and found a possible solution: Homebrew/legacy-homebrew#42209

Will try brew install xz

[yaaminiv]

Tried brew install xz, then configuring samtools and got the same error I've been getting.

Looked more into the Stack Overflow post linked in the Homebrew issue above: https://stackoverflow.com/questions/31612086/cannot-get-ffmpeg-to-work-after-installing-from-homebrew

Based on one of the suggestions, I ran brew search libmpfr (the library that wasn't loaded in any error above): Error: No formulae or casks found for "libmpfr".

Then ran brew search mpfr:

Warning: mpfr 4.1.0 is already installed and up-to-date.
To reinstall 4.1.0, run:
  brew reinstall mpfr

Going to reinstall mpfr with brew reinstall mpfr and see if that resolves the issue

[yaaminiv]

Reinstalling mpfr then running ./configure --prefix=/Users/Shared/Apps/samtools-1.12 lead to the same error.

Since mpfr is part of Homebrew, I realized I should just try installing samtools with brew install samtools! Pretty sure the installation worked.

[yaaminiv]

Solution: install samtools with Homebrew!