Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set webpack module type to "javascript/esm" when TS impliedNodeFormat is ESNext #1614

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
appendSuffixesIfMatch,
arrify,
formatErrors,
getImpliedNodeFormat,
isReferencedFile,
} from './utils';

Expand Down Expand Up @@ -147,9 +148,19 @@ function setModuleMeta(
// a previously cached version the TypeScript may be different and therefore should be
// treated as new
loaderContext._module!.buildMeta.tsLoaderFileVersion = fileVersion;
const impliedNodeFormat = getImpliedNodeFormat(
loaderContext._module!.resource,
andrewbranch marked this conversation as resolved.
Show resolved Hide resolved
instance,
loaderContext,
instance.program || instance.languageService?.getProgram() || instance.builderProgram?.getProgram()
);
if (impliedNodeFormat === /*ts.ModuleKind.ESNext*/ 99) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"I got 99 problems but EcmaScript modules ain't one"

loaderContext._module!.type = "javascript/esm";
}
}
}


/**
* Get a unique hash based on the contents of the options
* Hash is created from the values converted to strings
Expand Down Expand Up @@ -615,7 +626,7 @@ function getTranspilationEmit(
compilerOptions: { ...instance.compilerOptions, rootDir: undefined },
transformers: instance.transformers,
reportDiagnostics: true,
fileName,
fileName,
});
const module = loaderContext._module;

Expand Down
42 changes: 31 additions & 11 deletions src/instances.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
appendSuffixesIfMatch,
ensureProgram,
formatErrors,
getImpliedNodeFormat,
isReferencedFile,
makeError,
supportsSolutionBuild,
Expand Down Expand Up @@ -363,7 +364,7 @@ export function initializeInstance(
: instance.compiler.createProgram([], instance.compilerOptions));

const getProgram = () => program;
instance.transformers = getCustomTransformers(instance.loaderOptions, program, getProgram);
instance.transformers = getCustomTransformers(instance, loader, program, getProgram);
// Setup watch run for solution building
if (instance.solutionBuilderHost) {
addAssetHooks(loader, instance);
Expand Down Expand Up @@ -394,11 +395,12 @@ export function initializeInstance(
instance.builderProgram =
instance.watchOfFilesAndCompilerOptions.getProgram();

const getProgram = () => instance.builderProgram?.getProgram();
const getProgram = () => instance.builderProgram!.getProgram();
instance.program = getProgram();
instance.transformers = getCustomTransformers(
instance.loaderOptions,
instance.program,
instance,
loader,
instance.program!,
getProgram
);
} else {
Expand All @@ -414,8 +416,8 @@ export function initializeInstance(
instance.compiler.createDocumentRegistry()
);

const getProgram = () => instance.languageService!.getProgram();
instance.transformers = getCustomTransformers(instance.loaderOptions, getProgram(), getProgram);
const getProgram = () => instance.languageService!.getProgram()!;
instance.transformers = getCustomTransformers(instance, loader, getProgram(), getProgram);
}

addAssetHooks(loader, instance);
Expand All @@ -427,14 +429,25 @@ export function initializeInstance(
}
}

function getSetImpliedNodeFormatTransformer(instance: TSInstance, loaderContext: webpack.LoaderContext<LoaderOptions>, getProgram: () => typescript.Program) {
return (): typescript.Transformer<typescript.SourceFile> => {
return (sourceFile) => {
sourceFile.impliedNodeFormat = getImpliedNodeFormat(sourceFile.fileName, instance, loaderContext, getProgram());
return sourceFile;
}
}
}

export function getCustomTransformers(
loaderOptions: LoaderOptions,
program: typescript.Program | undefined,
getProgram: (() => typescript.Program | undefined) | undefined
instance: TSInstance,
loaderContext: webpack.LoaderContext<LoaderOptions>,
program: typescript.Program,
getProgram: (() => typescript.Program)
) {
// same strategy as https://github.com/s-panferov/awesome-typescript-loader/pull/531/files
const { loaderOptions } = instance;
let { getCustomTransformers: customerTransformers } = loaderOptions;
let getCustomTransformers = Function.prototype;
let getCustomTransformers;

if (typeof customerTransformers === 'function') {
getCustomTransformers = customerTransformers;
Expand All @@ -459,7 +472,14 @@ export function getCustomTransformers(
getCustomTransformers = customerTransformers;
}

return getCustomTransformers(program, getProgram);
let transformers = getCustomTransformers?.(program, getProgram);
if (loaderOptions.transpileOnly) {
(transformers ??= {}).before = [
getSetImpliedNodeFormatTransformer(instance, loaderContext, getProgram),
...(transformers?.before ?? []),
Copy link
Member

@johnnyreilly johnnyreilly Jun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this reminds me, I wonder whether we should bump the target for the emitted JavaScript of ts-loader at some point. I suspect it's quite an old version. "target": "es2018", - not too old

];
}
return transformers;
}

function getScriptRegexp(instance: TSInstance) {
Expand Down
5 changes: 4 additions & 1 deletion src/interfaces.ts
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ export interface TSInstance {
appendTsTsxSuffixesIfRequired: (filePath: string) => string;
loaderOptions: LoaderOptions;
rootFileNames: Set<string>;
moduleResolutionHost?: typescript.ModuleResolutionHost,
moduleResolutionCache?: ModuleResolutionCache;
typeReferenceResolutionCache?: typescript.TypeReferenceDirectiveResolutionCache;
/**
Expand All @@ -195,7 +196,7 @@ export interface TSInstance {
version: number;
dependencyGraph: DependencyGraph;
filesWithErrors?: TSFiles;
transformers: typescript.CustomTransformers;
transformers: typescript.CustomTransformers | undefined;
colors: Chalk;

otherFiles: TSFiles;
Expand Down Expand Up @@ -295,6 +296,8 @@ export interface TSFile {
project?: typescript.ResolvedProjectReference;
outputFileName?: string;
};
/** false implies a cached value of `undefined` */
impliedNodeFormat?: typescript.ModuleKind.CommonJS | typescript.ModuleKind.ESNext | false;
}

/** where key is filepath */
Expand Down
102 changes: 64 additions & 38 deletions src/servicesHost.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,21 @@ import {
useCaseSensitiveFileNames,
} from './utils';

function makeResolversAndModuleResolutionHost(
scriptRegex: RegExp,
loader: webpack.LoaderContext<LoaderOptions>,
export function makeModuleResolutionHost(
instance: TSInstance,
fileExists: (fileName: string) => boolean,
enableFileCaching: boolean
) {
const {
compiler,
compilerOptions,
appendTsTsxSuffixesIfRequired,
loaderOptions: {
resolveModuleName: customResolveModuleName,
resolveTypeReferenceDirective: customResolveTypeReferenceDirective,
},
} = instance;

loader: webpack.LoaderContext<LoaderOptions>,
enableFileCaching: boolean,
fileExists?: (fileName: string) => boolean
): ModuleResolutionHostMayBeCacheable {
const { files, otherFiles, compiler, compilerOptions, filePathKeyMapper } = instance;
fileExists ??= (fileName: string) => {
const filePathKey = filePathKeyMapper(fileName);
return (
files.has(filePathKey) ||
otherFiles.has(filePathKey) ||
compiler.sys.fileExists(fileName)
);
};
const newLine =
compilerOptions.newLine === constants.CarriageReturnLineFeedCode
? constants.CarriageReturnLineFeed
Expand All @@ -60,9 +58,6 @@ function makeResolversAndModuleResolutionHost(
// loader.context seems to work fine on Linux / Mac regardless causes problems for @types resolution on Windows for TypeScript < 2.3
const getCurrentDirectory = () => loader.context;

// make a (sync) resolver that follows webpack's rules
const resolveSync = makeResolver(loader._compiler!.options);

const moduleResolutionHost: ModuleResolutionHostMayBeCacheable = {
trace: logData => instance.log.log(logData),
fileExists,
Expand All @@ -79,21 +74,22 @@ function makeResolversAndModuleResolutionHost(
getDefaultLibFileName: options => compiler.getDefaultLibFilePath(options),
};


if (enableFileCaching) {
addCache(moduleResolutionHost);
}

return makeResolvers(
compiler,
compilerOptions,
moduleResolutionHost,
customResolveTypeReferenceDirective,
customResolveModuleName,
resolveSync,
appendTsTsxSuffixesIfRequired,
scriptRegex,
instance
);

if (!instance.moduleResolutionCache && !instance.loaderOptions.resolveModuleName) {
instance.moduleResolutionCache = createModuleResolutionCache(
instance,
moduleResolutionHost
);
}

instance.moduleResolutionHost = moduleResolutionHost;

return moduleResolutionHost;

function readFile(
filePath: string,
Expand Down Expand Up @@ -134,6 +130,42 @@ function makeResolversAndModuleResolutionHost(
}
}

function makeResolversAndModuleResolutionHost(
scriptRegex: RegExp,
loader: webpack.LoaderContext<LoaderOptions>,
instance: TSInstance,
fileExists: (fileName: string) => boolean,
enableFileCaching: boolean
) {
const {
compiler,
compilerOptions,
appendTsTsxSuffixesIfRequired,
loaderOptions: {
resolveModuleName: customResolveModuleName,
resolveTypeReferenceDirective: customResolveTypeReferenceDirective,
},
} = instance;



// make a (sync) resolver that follows webpack's rules
const resolveSync = makeResolver(loader._compiler!.options);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need this? Based on logic it is just an error - https://github.com/TypeStrong/ts-loader/blob/main/src/resolver.ts#L11

Webpack has great API for resolving, you can use (just an example of code, we should have two resolvers for cjs and esm in the real life):

const resolve = loaderContext.getResolve({
    dependencyType: "typescript",
    conditionNames: ["types", "..."],
    mainFiles: ["index", "..."],
    extensions: [".ts", ".cts", "..."],
});

And hide a lot of options for basic resolve under the hood - less options better DX, but yeah, I think it should be done in another PR, here is just about ESM

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just moved code, not code I wrote

const moduleResolutionHost = makeModuleResolutionHost(instance, loader, enableFileCaching, fileExists);

return makeResolvers(
compiler,
compilerOptions,
moduleResolutionHost,
customResolveTypeReferenceDirective,
customResolveModuleName,
resolveSync,
appendTsTsxSuffixesIfRequired,
scriptRegex,
instance
);
}

/**
* Create the TypeScript language service
*/
Expand Down Expand Up @@ -768,7 +800,7 @@ export function makeSolutionBuilderHost(
);

// Keeps track of the various `typescript.CustomTransformers` for each program that is created.
const customTransformers = new Map<string, typescript.CustomTransformers>();
const customTransformers = new Map<string, typescript.CustomTransformers | undefined>();

// let lastBuilderProgram: typescript.CreateProgram | undefined = undefined;
const solutionBuilderHost: SolutionBuilderWithWatchHost = {
Expand Down Expand Up @@ -802,7 +834,7 @@ export function makeSolutionBuilderHost(
if (typeof project === "string") {
// Custom transformers need a reference to the `typescript.Program`, that reference is
// unavailable during the the `getCustomTransformers` callback below.
const transformers = getCustomTransformers(instance.loaderOptions, result.getProgram(), result.getProgram);
const transformers = getCustomTransformers(instance, loader, result.getProgram(), result.getProgram);
customTransformers.set(project, transformers);
}
}
Expand Down Expand Up @@ -1335,12 +1367,6 @@ function makeResolveModuleName(
instance: TSInstance
): ResolveModuleName {
if (customResolveModuleName === undefined) {
if (!instance.moduleResolutionCache) {
instance.moduleResolutionCache = createModuleResolutionCache(
instance,
moduleResolutionHost
);
}
return (
moduleName,
containingFileName,
Expand Down
35 changes: 35 additions & 0 deletions src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
TSInstance,
} from './interfaces';
import { getInputFileNameFromOutput } from './instances';
import { makeModuleResolutionHost } from './servicesHost';
/**
* The default error formatter.
*/
Expand Down Expand Up @@ -350,3 +351,37 @@ export function useCaseSensitiveFileNames(
? loaderOptions.useCaseSensitiveFileNames
: compiler.sys.useCaseSensitiveFileNames;
}

export function toPath(
absoluteFileName: string,
compiler: typeof typescript,
loaderOptions: LoaderOptions
): typescript.Path {
if (!path.isAbsolute(absoluteFileName)) {
throw new Error(`Expected argument to 'toPath' to be an absolute filename. Received '${absoluteFileName}'.`);
}
if (useCaseSensitiveFileNames(compiler, loaderOptions)) {
return absoluteFileName as typescript.Path;
}
return absoluteFileName.toLowerCase() as typescript.Path;
}

export function getImpliedNodeFormat(fileName: string, instance: TSInstance, loaderContext: webpack.LoaderContext<LoaderOptions>, program?: typescript.Program) {
const file = instance.files.get(instance.filePathKeyMapper(fileName));
if (file && file.impliedNodeFormat !== undefined) {
return file.impliedNodeFormat || undefined;
}
const path = toPath(fileName, instance.compiler, instance.loaderOptions);
const sourceFile = program?.getSourceFileByPath(path);
const impliedNodeFormat = sourceFile
? sourceFile.impliedNodeFormat
: instance.compiler.getImpliedNodeFormatForFile(
path,
instance.moduleResolutionCache?.getPackageJsonInfoCache(),
instance.moduleResolutionHost ??= makeModuleResolutionHost(instance, loaderContext, instance.loaderOptions.experimentalFileCaching),
instance.compilerOptions);
if (file) {
file.impliedNodeFormat = impliedNodeFormat ?? false;
}
return impliedNodeFormat;
}
4 changes: 4 additions & 0 deletions test/comparison-tests/esm_mts/app.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
import assert from "assert";
import externalLib from "./lib/externalLib.js";
assert.deepStrictEqual(externalLib, { default: {} });
console.log("PASS");
Loading