* Court spokeswoman says Thomas likely has the flu
* Justice Ginsburg looks vigorous after cancer treatment
* Major LGBT rights case to be argued on Tuesday (Adds details on insanity defense argument)
By Lawrence Hurley and Andrew Chung
WASHINGTON, Oct 7 (Reuters) - The U.S. Supreme Court on Monday debated the insanity defense in criminal prosecutions and delivered a potential setback to gun control advocates as it opened its new term on Monday with a flurry of activity on a day when its longest-serving justice, Clarence Thomas, was absent due to illness.
The conservative-majority court returned after a three-month break to launch its new nine-month term with arguments in a trio of cases. The court hears the term's first major case on Tuesday on whether a landmark federal anti-discrimination law that bars sex discrimination in the workplace protects gay and transgender employees.
In remarks from the bench before the first argument, Chief Justice John Roberts said Thomas was "indisposed" due to illness but would participate in deciding the cases argued on Monday. A court spokeswoman said Thomas, 71, likely has the flu. One of the nine-member court's most conservative justices, Thomas has served since 1991.
The court said it would not cancel its scheduled Dec. 2 arguments, as requested by New York City, in a lawsuit by gun owners and the state's National Rifle Association affiliate challenging restrictions on handgun owners transporting firearms outside the home.
The city had asked the justices to ditch the arguments because its measure was recently amended, meaning there was no longer any reason to hear the dispute. Hearing the case could enable the court's conservative majority to rule against a gun control measure, although the justices also still could dismiss it after the arguments.
During Monday's first argument, the justices wrestled with the insanity defense, which bars holding criminally responsible mentally impaired defendants who do not know right from wrong. They seemed divided over a Kansas law, enacted in 1996, that prevents defendants from arguing insanity. The law lets defendants argue that, due to mental illness, they could not have intended to commit the crime.
Liberal Justice Ruth Bader Ginsburg, 86, asked the first question and appeared vigorous after undergoing radiation therapy in August to treat a cancerous tumor on her pancreas.
The case involved James Kraig Kahler, sentenced to death in Kansas for the 2009 fatal shootings of four family members. He shot his wife and two teenage daughters with a high-powered rifle at the home of his wife's grandmother, who he also killed. He spared his son, who was 9 years old.
Kahler was diagnosed with major depression and obsessive personality disorder. A psychiatric expert hired by the defense said Kahler's illness was so severe he "lost control" of his actions.
In Kansas, he could not claim he was not guilty by reason of insanity. He challenged the measure as a violation of the U.S. Constitution's 14th Amendment guarantee of due process under the law and the Eighth Amendment prohibition on cruel and unusual punishment. His conviction was upheld by the state's top court.
Conservative Justice Samuel Alito said it has been calculated that one in five Americans has some sort of mental disorder. "All of them could go to the jury on the question of whether they had the capacity to know that what they were doing when they committed the crime was morally wrong," Alito said.
Several justices pointed to the long history of the insanity defense in U.S. law. "If it's historically established that you cannot punish people who don't know the difference between right and wrong, that certainly sounds like something that is rooted in the conscience and would be ranked as fundamental," said Chief Justice John Roberts, a conservative.
Idaho, Montana and Utah also have abandoned the insanity defense. Other states, the District of Columbia and the federal criminal justice system allow it.
A ruling is due by the end of June.
The other two arguments on Monday focused on whether the Constitution requires unanimous jury verdicts and on fees in patent litigation.
In the jury verdicts dispute, the court appeared likely to rule in favor of Louisiana inmate Evangelisto Ramos, who was convicted on a 10-2 jury vote in a 2014 murder. The court is weighing whether Constitution's Sixth Amendment guarantee of trial by jury requires a unanimous verdict.
Only two states, Louisiana and Oregon, had allowed non-unanimous verdicts. Louisiana began prohibiting non-unanimous verdicts starting this past January, but that change did not apply retroactively.
The justices turned away appeals in several business-related cases. They let a discrimination lawsuit proceed against Domino's Pizza Inc by a blind man who accused it of violating a law barring discrimination against people with disabilities after he was unable to place an order online.
The court spurned Amazon.com's bid to avoid a lawsuit concerning overtime pay for workers who face lengthy post-shift security screenings.
It also rejected appeals in patent disputes including one involving Apple Inc and another involving Acorda Therapeutics Inc.
The court declined to hear Las Vegas sports gambler William "Billy" Walters' appeal of a 2017 insider trading conviction that landed him a five-year prison sentence in a case that also drew attention because of his ties to billionaire investor Carl Icahn and golfer Phil Mickelson.
Reporting by Larence Hurley and Andrew Chung; Editing by Will Dunham