
\begin{DoxyItemize}
\item Added\+: Support for ES2018. The only change needed was recognizing the {\ttfamily s} regex flag.
\item Changed\+: {\itshape All} tokens returned by the {\ttfamily match\+To\+Token} function now have a {\ttfamily closed} property. It is set to {\ttfamily undefined} for the tokens where “closed” doesn’t make sense. This means that all tokens objects have the same shape, which might improve performance.
\end{DoxyItemize}

These are the breaking changes\+:


\begin{DoxyItemize}
\item `'/a/s\textquotesingle{}.match(js\+Tokens){\ttfamily no longer returns}\mbox{[}\textquotesingle{}/\textquotesingle{}, \textquotesingle{}a\textquotesingle{}, \textquotesingle{}/\textquotesingle{}, \textquotesingle{}s\textquotesingle{}\mbox{]}{\ttfamily , but }\mbox{[}\textquotesingle{}/a/s\textquotesingle{}\mbox{]}{\ttfamily . (There are of course other variations of this.)}
\item {\ttfamily Code that rely on some token objects not having the}closed\`{} property could now behave differently.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14002}{}\doxysubsection{Version 3.\+0.\+2 (2017-\/06-\/28)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14002}

\begin{DoxyItemize}
\item No code changes. Just updates to the readme.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14003}{}\doxysubsection{Version 3.\+0.\+1 (2017-\/01-\/30)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14003}

\begin{DoxyItemize}
\item Fixed\+: ES2015 unicode escapes with more than 6 hex digits are now matched correctly.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14004}{}\doxysubsection{Version 3.\+0.\+0 (2017-\/01-\/11)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14004}
This release contains one breaking change, that should \href{http://v8project.blogspot.se/2017/01/speeding-up-v8-regular-expressions.html}{\texttt{ improve performance in V8}}\+:

\begin{quote}
So how can you, as a Java\+Script developer, ensure that your Reg\+Exps are fast? If you are not interested in hooking into Reg\+Exp internals, make sure that neither the Reg\+Exp instance, nor its prototype is modified in order to get the best performance\+:


\begin{DoxyCode}{0}
\DoxyCodeLine{var re = /./g;}
\DoxyCodeLine{re.exec('');  // Fast path.}
\DoxyCodeLine{re.new\_property = 'slow'; }

\end{DoxyCode}
 \end{quote}
This module used to export a single regex, with {\ttfamily .match\+To\+Token} bolted on, just like in the above example. This release changes the exports of the module to avoid this issue.

Before\+:


\begin{DoxyCode}{0}
\DoxyCodeLine{import jsTokens from "{}js-\/tokens"{}}
\DoxyCodeLine{// or:}
\DoxyCodeLine{var jsTokens = require("{}js-\/tokens"{})}
\DoxyCodeLine{var matchToToken = jsTokens.matchToToken }

\end{DoxyCode}


After\+:


\begin{DoxyCode}{0}
\DoxyCodeLine{import jsTokens, \{matchToToken\} from "{}js-\/tokens"{}}
\DoxyCodeLine{// or:}
\DoxyCodeLine{var jsTokens = require("{}js-\/tokens"{}).default}
\DoxyCodeLine{var matchToToken = require("{}js-\/tokens"{}).matchToToken }

\end{DoxyCode}
\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14005}{}\doxysubsection{Version 2.\+0.\+0 (2016-\/06-\/19)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14005}

\begin{DoxyItemize}
\item Added\+: Support for ES2016. In other words, support for the {\ttfamily $\ast$$\ast$} exponentiation operator.
\end{DoxyItemize}

These are the breaking changes\+:


\begin{DoxyItemize}
\item `'$\ast$$\ast$\textquotesingle{}.match(js\+Tokens){\ttfamily no longer returns}\mbox{[}\textquotesingle{}$\ast$\textquotesingle{}, \textquotesingle{}$\ast$\textquotesingle{}\mbox{]}{\ttfamily , but}\mbox{[}\textquotesingle{}$\ast$$\ast$\textquotesingle{}\mbox{]}{\ttfamily . -\/}\textquotesingle{}$\ast$$\ast$=\textquotesingle{}.match(js\+Tokens){\ttfamily no longer returns}\mbox{[}\textquotesingle{}$\ast$\textquotesingle{}, \textquotesingle{}$\ast$=\textquotesingle{}\mbox{]}{\ttfamily , but}\mbox{[}\textquotesingle{}$\ast$$\ast$=\textquotesingle{}\mbox{]}\`{}.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14006}{}\doxysubsection{Version 1.\+0.\+3 (2016-\/03-\/27)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14006}

\begin{DoxyItemize}
\item Improved\+: Made the regex ever so slightly smaller.
\item Updated\+: The readme.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14007}{}\doxysubsection{Version 1.\+0.\+2 (2015-\/10-\/18)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14007}

\begin{DoxyItemize}
\item Improved\+: Limited npm package contents for a smaller download. Thanks to @zertosh!
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14008}{}\doxysubsection{Version 1.\+0.\+1 (2015-\/06-\/20)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14008}

\begin{DoxyItemize}
\item Fixed\+: Declared an undeclared variable.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14009}{}\doxysubsection{Version 1.\+0.\+0 (2015-\/02-\/26)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14009}

\begin{DoxyItemize}
\item Changed\+: Merged the \textquotesingle{}operator\textquotesingle{} and \textquotesingle{}punctuation\textquotesingle{} types into \textquotesingle{}punctuator\textquotesingle{}. That type is now equivalent to the Punctuator token in the ECMAScript specification. (Backwards-\/incompatible change.)
\item Fixed\+: A {\ttfamily -\/} followed by a number is now correctly matched as a punctuator followed by a number. It used to be matched as just a number, but there is no such thing as negative number literals. (Possibly backwards-\/incompatible change.)
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14010}{}\doxysubsection{Version 0.\+4.\+1 (2015-\/02-\/21)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14010}

\begin{DoxyItemize}
\item Added\+: Support for the regex {\ttfamily u} flag.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14011}{}\doxysubsection{Version 0.\+4.\+0 (2015-\/02-\/21)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14011}

\begin{DoxyItemize}
\item Improved\+: {\ttfamily js\+Tokens.\+match\+To\+Token} performance.
\item Added\+: Support for octal and binary number literals.
\item Added\+: Support for template strings.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14012}{}\doxysubsection{Version 0.\+3.\+1 (2015-\/01-\/06)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14012}

\begin{DoxyItemize}
\item Fixed\+: Support for unicode spaces. They used to be allowed in names (which is very confusing), and some unicode newlines were wrongly allowed in strings and regexes.
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14013}{}\doxysubsection{Version 0.\+3.\+0 (2014-\/12-\/19)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14013}

\begin{DoxyItemize}
\item Changed\+: The {\ttfamily js\+Tokens.\+names} array has been replaced with the {\ttfamily js\+Tokens.\+match\+To\+Token} function. The capturing groups of {\ttfamily js\+Tokens} are no longer part of the public API; instead use said function. See this \href{https://gist.github.com/lydell/be49dbf80c382c473004}{\texttt{ gist}} for an example. (Backwards-\/incompatible change.)
\item Changed\+: The empty string is now considered an “invalid” token, instead an “empty” token (its own group). (Backwards-\/incompatible change.)
\item Removed\+: component support. (Backwards-\/incompatible change.)
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14014}{}\doxysubsection{Version 0.\+2.\+0 (2014-\/06-\/19)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14014}

\begin{DoxyItemize}
\item Changed\+: Match ES6 function arrows ({\ttfamily =$>$}) as an operator, instead of its own category (“function\+Arrow”), for simplicity. (Backwards-\/incompatible change.)
\item Added\+: ES6 splats ({\ttfamily ...}) are now matched as an operator (instead of three punctuations). (Backwards-\/incompatible change.)
\end{DoxyItemize}\hypertarget{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14015}{}\doxysubsection{Version 0.\+1.\+0 (2014-\/03-\/08)}\label{md_sfcode_node_modules_js_tokens_CHANGELOG_autotoc_md14015}

\begin{DoxyItemize}
\item Initial release. 
\end{DoxyItemize}