The Ampersand

What is the ampersand? The ampersand (or And-sign, '&') is used for conditinal execution in batch scripts. So ECHO one & ECHO two will give you the output

The problem with the ampersand is, that it is a special character that can occur in paths and filenames (other characters like >, <, : etc. are not allowed!) passed to batched scripts. Of course, these characters can occur in user input, when reading text files etc, but that's not the point here because I want to focus on the ampersand only.
When it occurs in a batch script parameter, it will be either quoted or escaped:

Batch script parameters

You can refer to parameters to your batch script with %1 to %9. You will get the parameter just as it was entered. You will not "see" the carets (^) of an escaped string. So if the first parameter entered is "C:\files & more\file.txt", an ECHO will give you exactly that phrase:
results in

You can also assign the parameter's value to an environment variable with the statement SET PARAM=%1. But what can we do about the annoying quotes around the string? The batch allows us multiple parameter substitutions, which all look like %~[op]n. So %~d1 gives us the drive name of the first parameter regarding it as an absolute or relative file path. All substitutions remove the optional surrounding quotes; if you just want to remove the quotes, you use %~1 without any operator.

But if the string contains an escaped ampersand or you remove the surrounding quotes from a string with an ampersand, that character will be treated by the interpretor as the conditional command execution character which will obviously break you script.

Parsing and executing a batch script

The way the batch interpretor works itself through a batch file is very special. In short this looks like this:

1. Read a line. Round brackets spanning lines count as a single line.
2. Expand all environment variables and batch script parameters by replacing them through their values. Optional substitution operations can be performed.
3. Parse the line, identify commands, strings, operators etc.
4. Perform delayed variable expansion on environment variables if enabled. Optional substitution operations can be performed.
5. Execute all identified commands in order.

I assume you are already familiar with round brackets and substitution operations, so I will only explain the delayed variable expansion. Variables to be delayed expanded are written with exclamation marks instead of percent signs:
Because the expansion takes place AFTER parsing, all special characters (including ampersand, caret, pipe etc.) will be indirectly escaped. That's why you can use this technique to handle strings with ampersands!
You enable delayed variable expansion with

Working with Variables

As I told above, parameters with ampersand easily break your script. So you have to either quote or escape your parameters when using them. Just - you cannot escape parameters in a script, that only works for variables! Alright, let's quote - but what if a parameter is already quoted?
I told you above that through parameter substitution you can remove the quotes. With %~1 you are left with an unquoted string possibly containing one ore ampersands waiting to break your script. So ALWAYS use something like

Okay, now you have a quoted string - but that sometimes sucks like in the ECHO line, because that command also echoes the quotes.
Aaaahh, yes, I wrote something about substitution operations.
That removes the first and the last character, which are my manually added quotes. But dammit, it's unquoted now which breaks the script again! Hmmm, wait, there was something about "delayed" variable expansion after parsing? Right, that's the way to use unquoted strings WITH ampersands! As the variables are expanded AFTER parsing, special characters won't break your code because they will always appear as strings.
Assignment works as well:
An now we got it: The variable STR contains the unquoted input parameter string. So if you want to use that variable, you either have to quote it again or use delayed expansion:

Hmm, but what about when one actually cannot use delayed expansion? For example when using subroutines with local variables of which one should be left after return?
This special technique to allow both local and global context needs two SET statements (note the missing space between quote and ampersand!!!)? That looks really bad, but is in fact the only clean solution.
However, if you are sure that the only special character contained in that string is the ampersand (e.g. it is an original file path), you can manually escape that string:
That has the same effect like above, but only for the ampersand!

Notes

• The %0 parameter (storing the script name) can also be placed in a path containing ampersands. So the rules stated above also apply to this parameter.
• Avoid using %*. It merges all parameters into one single string. That means, it can contain quoted parameters, so you cannot quote %*, and it can contain unquoted, escaped parameters, so you must use quotes. You see, there is no way handling it correctly, so you should use the shift command instead.
• A common IF expression is
  
  IF "%TEST%" == "abcd" GOTO :DO_IT
  
  This works unless the variable contains double quotes. You could use other characters like [] to surround your variables in case they are empty, however this would still break in case of an ampersand. So here, you can use two equitable construtions: escape the quotes or use delayed variable expansion:
  
  IF "%TEST:"=^"%" == "abcd" GOTO :DO_IT
  IF "!TEST!" == "abcd" GOTO :DO_IT
  
• As for FOR constructs, you should be very careful and use delayed expansion and quotation whenever possible. Sometimes you have to work the right solution out by trial and error.