Skip to content
Snippets Groups Projects
Unverified Commit 0dc75042 authored by Romeo Van Snick's avatar Romeo Van Snick Committed by GitHub
Browse files

Combine columns (#42021)

* [MLv2] Add `preview-expression` to eval an expression on sample data

This is needed to show the previews in the UX for the combining columns
epic #39977.

Fixes #39979.

* [MBQL lib] Add "Combine columns" drill-thru

This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.

See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569)
for discussion of how we might approach that issue.

Fixes #39978.

* [MBQL lib] Add "Combine columns" drill-thru

This is following the original plan of using a drill for this. It's hard
to combine that with the "preview" functionality.

See [thread](https://metaboat.slack.com/archives/C06P22KS4JH/p1712264174056569

)
for discussion of how we might approach that issue.

Fixes #39978.

* [MLv2] Add `preview-expression` to eval an expression on sample data

This is needed to show the previews in the UX for the combining columns
epic #39977.

Fixes #39979.

* [MLv2] Add `preview-expression` to eval an expression on sample data

This is needed to show the previews in the UX for the combining columns
epic #39977.

Fixes #39979.

* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"

This reverts commit dac17f84aaae09906a206106290f7f58dea659bd, reversing
changes made to 2934fab19db495ca8104801d41bc8aba457e401f.

* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"

This reverts commit 2934fab19db495ca8104801d41bc8aba457e401f, reversing
changes made to 6a43c22e78fb616c7ac0b024811b2989a1ae0f20.

* Revert "Merge branch 'mblib-preview-expressions' into combine-columns"

This reverts commit 6a43c22e78fb616c7ac0b024811b2989a1ae0f20, reversing
changes made to 2257333a8ae69f0c9a6f6fdf0c8f5d837263f97a.

* [FE] "Combine columns" drill thru (#40082)

* Type Section['icon'] properly, including the hack for summarize section

* Add combine columns section

* Add types for combine columns drill

* Add combine columns drill component stub

* Mock combine columns drill

* Add component stub

* Encapsulate typing hacks

* Extract CombineColumnsDrill

* Extract ColumnAndSeparatorRow

* Improve naming, add button to edit separators

* Translate string

* Improve styling

* Separator input

* Handle removing columns

* Rename lib.ts to utils.ts

* Implement onSubmit

* Add getNextColumnAndSeparator

* Use form for a11y

* Handle vertical overflow

* Handle horizontal overflow

* Extract formatSeparator

* Use proper translations

* Adjust min/max sizes

* Fix scrollbar being unnecessarily shown

* Display source column name

* Avoid serializing the entire column and putting it into DOM

* Use module.css extension

* Format code

* Improve a11y

* Mock combineColumnsDrillExpression and previewExpression

* Add Preview component

* Move preview logic to Preview component

* Style Preview

* Format code

* Move styles to CSS modules

* Update UI to latest designs
- move "Add another column" button
- remove "Separated by X" button
- always show separator inputs

* Remove displayInfo mock

* Add asReturned

* Update CombineColumnsDrillThruInfo

* Integrate new way of working with combine columns drill

* Fix generating expression names

* Integrate drill with click actions

* Integrate new preview workflow

* Revert "Update UI to latest designs"

This reverts commit cda039dffe9d452c0866f63800e2e887a540fe67.

* Use previewExpression interface correctly

* Use new preview_expression API

* Make sure columns are created with original query and stage index

* Extract usePreview and handle preview errors

* Update popover title

* Move add column button

* Fix outline being cut off

* Style preview label as per design

* Format empty separator

* Make preview scrollable

* Use project convention

* Simplify ScrollArea usage

* Use ScrollArea in CombineColumnsDrill

* Update comment

* Remove Lib.previewExpression

* Clean up the preview after removing Lib.previewExpression

* Implement getPreview

* Change default column preview to 'text'

* Remove usePreview hook which is no longer in use

* Rename Preview to Example

* Use different placeholder for separator input

* Add whitespace placeholder

* Add ColumnPicker boilerplate

* Export Input

* Use QueryColumnPicker for ColumnPicker

* Pass width to AccordionList

* Add sequences popover events with tippy

* Remove unused ColumnOption helpers

* Add testid to Example

* Add e2e test for combine column in header

* Select content of separator input on focus

* Reduce padding on example

* Add monospace variant to Text

* Use monospace variant in Example

* Use monospace font in textinput

* Match faux-select label styles to our TextInput label styles

* Add missing color

* Remove !important

* Use pre whitespace so consecutive spaces show up

* Remove empty default

* Add example for boolean

* Remove !important

* Use should.have text over contain

---------

Co-authored-by: default avatarRomeo Van Snick <romeo@romeovansnick.be>

* Remove duplicated code

* Export useMantineTheme from metabase/ui

* Use useMantineTheme instead of reading theme directly

* Remove setTimeout in popover

* Use plain t instead of jt

* Move Popover dropdown sequence hack to metabase/ui

---------

Co-authored-by: default avatarBraden Shepherdson <braden@metabase.com>
Co-authored-by: default avatarKamil Mielnik <kamil@kamilmielnik.com>
parent 9887f222
No related merge requests found
Showing
with 665 additions and 4 deletions
......@@ -22,9 +22,11 @@ and documented in this changelog.
`as-returned` looks at the query and stage, and shifts to a later stage if necessary. If a later stage is needed but
we were already on the last stage, a new empty stage is appended.
- New functions `column-extractions`, `extract`, and `extraction-expression` have been added.
- `column-extractions` returns a list of _extractions_, which are possible custom expressions we can derive from a
given column. For example, getting the host or base domain name from a URL or email address, or the day of the week
from a date or datetime.
- `extract` applies an extraction to the query.
- `extraction-expression` returns the expression for the extraction, allowing further editing.
import { openPeopleTable, popover, restore } from "e2e/support/helpers";
describe("scenarios > visualizations > drillthroughs > table_drills > combine columns", () => {
beforeEach(() => {
restore();
cy.signInAsAdmin();
});
it("should be possible to combine columns from the a table header", () => {
openPeopleTable({ limit: 3, mode: "notebook" });
cy.findByLabelText("Pick columns").click();
popover().within(() => {
cy.findByText("Select none").click();
cy.findByLabelText("Email").click();
});
cy.findByLabelText("Pick columns").click();
cy.button("Visualize").click();
cy.findAllByTestId("header-cell").contains("Email").click();
popover().findByText("Combine columns").click();
popover().within(() => {
cy.findByTestId("combine-column-example").should(
"contain",
"email@example.com12345",
);
cy.findByText("ID").click();
});
popover()
.last()
.within(() => {
cy.findByText("Name").click();
});
popover().within(() => {
cy.findByText("Separated by (empty)").click();
cy.findByLabelText("Separator").type("__");
cy.findByTestId("combine-column-example").should(
"have.text",
"email@example.com__text",
);
cy.findByText("Add column").click();
cy.findByTestId("combine-column-example").should(
"have.text",
"email@example.com__text__12345",
);
cy.findAllByRole("textbox").last().clear();
cy.findByTestId("combine-column-example").should(
"have.text",
"email@example.com__text12345",
);
cy.findAllByRole("textbox").last().clear().type("+");
cy.findByTestId("combine-column-example").should(
"have.text",
"email@example.com__text+12345",
);
cy.findByText("Done").click();
});
cy.findAllByTestId("header-cell")
.last()
.should("have.text", "Email Name ID");
});
});
......@@ -2,13 +2,13 @@ import * as ML from "cljs/metabase.lib.js";
import type { DatasetColumn, RowValue } from "metabase-types/api";
import type {
FilterDrillDetails,
ColumnMetadata,
ClickObjectDataRow,
ClickObjectDimension,
ColumnMetadata,
DrillThru,
Query,
FilterDrillDetails,
PivotType,
Query,
} from "./types";
// NOTE: value might be null or undefined, and they mean different things!
......
......@@ -430,6 +430,7 @@ export type DrillThruType =
| "drill-thru/automatic-insights"
| "drill-thru/column-extract"
| "drill-thru/column-filter"
| "drill-thru/combine-columns"
| "drill-thru/distribution"
| "drill-thru/fk-details"
| "drill-thru/fk-filter"
......@@ -463,6 +464,9 @@ export type ColumnExtractDrillThruInfo =
extractions: ColumnExtractionInfo[];
};
export type CombineColumnsDrillThruInfo =
BaseDrillThruInfo<"drill-thru/combine-columns">;
export type QuickFilterDrillThruOperator =
| "="
| ""
......@@ -528,6 +532,7 @@ export type ZoomTimeseriesDrillThruInfo =
export type DrillThruDisplayInfo =
| ColumnExtractDrillThruInfo
| CombineColumnsDrillThruInfo
| QuickFilterDrillThruInfo
| PKDrillThruInfo
| ZoomDrillThruInfo
......
......@@ -66,8 +66,8 @@ export function QueryColumnPicker({
checkIsColumnSelected,
onSelect,
onClose,
"data-testid": dataTestId,
width,
"data-testid": dataTestId,
hasInitialFocus = true,
alwaysExpanded,
disableSearch,
......
import { t } from "ttag";
import type {
ClickActionPopoverProps,
Drill,
} from "metabase/visualizations/types/click-actions";
import * as Lib from "metabase-lib";
import { CombineColumnsDrill } from "./components";
export const combineColumnsDrill: Drill<Lib.CombineColumnsDrillThruInfo> = ({
question,
query,
stageIndex,
clicked,
}) => {
if (!clicked.column) {
return [];
}
const column = Lib.fromLegacyColumn(query, stageIndex, clicked.column);
const DrillPopover = ({
onChangeCardAndRun,
onClose,
}: ClickActionPopoverProps) => (
<CombineColumnsDrill
column={column}
query={query}
stageIndex={stageIndex}
onSubmit={newQuery => {
const nextQuestion = question.setQuery(newQuery);
const nextCard = nextQuestion.card();
onChangeCardAndRun({ nextCard });
onClose();
}}
/>
);
return [
{
name: "combine",
title: t`Combine columns`,
section: "combine",
icon: "add",
buttonType: "horizontal",
popover: DrillPopover,
},
];
};
.separator {
flex: 0 0 auto;
}
.column {
flex: 1;
}
button.remove {
flex: 0 0 auto;
border-color: transparent;
}
.whitespacePlaceholder {
pointer-events: none;
user-select: none;
}
import { useState, type FocusEvent } from "react";
import { t } from "ttag";
import {
Box,
Button,
Flex,
Icon,
Text,
TextInput,
useMantineTheme,
} from "metabase/ui";
import type * as Lib from "metabase-lib";
import type { ColumnAndSeparator } from "../../types";
import { formatSeparator } from "../../utils";
import { ColumnPicker } from "../ColumnPicker";
import S from "./ColumnAndSeparatorRow.module.css";
interface Props {
query: Lib.Query;
stageIndex: number;
column: Lib.ColumnMetadata;
columns: Lib.ColumnMetadata[];
index: number;
separator: string;
showLabels: boolean;
showRemove: boolean;
showSeparator: boolean;
onChange: (index: number, change: Partial<ColumnAndSeparator>) => void;
onRemove: (index: number) => void;
}
export const ColumnAndSeparatorRow = ({
query,
stageIndex,
column,
columns,
index,
separator,
showLabels,
showRemove,
showSeparator,
onChange,
onRemove,
}: Props) => {
const [isFocused, setIsFocused] = useState(false);
const { fontFamilyMonospace } = useMantineTheme();
function handleFocus(event: FocusEvent<HTMLInputElement>) {
setIsFocused(true);
event.target.select();
}
return (
<Flex align="flex-end" gap={12}>
{showSeparator && (
<Box pos="relative">
<TextInput
className={S.separator}
label={showLabels ? t`Separator` : undefined}
placeholder={formatSeparator("")}
value={separator}
w={110}
onChange={event => {
const separator = event.target.value;
onChange(index, { separator });
}}
onBlur={() => setIsFocused(false)}
onFocus={handleFocus}
styles={{
input: {
fontFamily: fontFamilyMonospace as string,
},
}}
/>
{separator === " " && !isFocused && (
<Text
bottom={8} // using bottom instead of top because the input does not always have a label
className={S.whitespacePlaceholder}
color="text-light"
left={1} // account for TextInput border
pos="absolute"
px="0.6875rem" // same as TextInput
size="md"
unselectable="on"
>
{formatSeparator(separator)}
</Text>
)}
</Box>
)}
<Box className={S.column}>
<ColumnPicker
query={query}
stageIndex={stageIndex}
columns={columns}
label={showLabels ? t`Column` : undefined}
value={column}
onChange={column => {
onChange(index, { column });
}}
/>
</Box>
{showRemove && (
<Button
classNames={{
root: S.remove,
}}
aria-label={t`Remove column`}
leftIcon={<Icon name="close" />}
variant="default"
onClick={() => {
onRemove(index);
}}
/>
)}
</Flex>
);
};
export * from "./ColumnAndSeparatorRow";
div.button {
justify-content: space-between;
font-weight: normal;
}
button.root {
&:hover {
background: none;
}
&:hover .button {
color: var(--color-text-dark);
}
&.open,
&:focus-visible {
border-color: var(--color-brand);
outline: none;
}
}
import classNames from "classnames";
import type { MouseEvent, KeyboardEvent, FocusEvent } from "react";
import { useRef, useState, useMemo } from "react";
import { t } from "ttag";
import { QueryColumnPicker } from "metabase/common/components/QueryColumnPicker";
import { color } from "metabase/lib/colors";
import {
Button,
Icon,
Input,
Popover,
FocusTrap,
useMantineTheme,
} from "metabase/ui";
import * as Lib from "metabase-lib";
import styles from "./ColumnPicker.module.css";
type ColumnInputProps = {
query: Lib.Query;
stageIndex: number;
columns: Lib.ColumnMetadata[];
label?: string;
value: Lib.ColumnMetadata;
onChange: (column: Lib.ColumnMetadata) => void;
};
export function ColumnPicker({
query,
stageIndex,
columns,
label,
value,
onChange,
}: ColumnInputProps) {
const theme = useMantineTheme();
const columnGroups = useMemo(() => Lib.groupColumns(columns), [columns]);
const [open, setOpen] = useState(false);
const button = useRef<HTMLButtonElement>(null);
function handleOpen() {
setOpen(true);
}
function handleClose() {
setOpen(false);
button.current?.focus();
}
function handleBlur(evt: FocusEvent<HTMLDivElement>) {
if (!evt.currentTarget || !evt.relatedTarget) {
return;
}
if (!evt.currentTarget.contains(evt.relatedTarget as Node)) {
setOpen(false);
}
}
function handleButtonClick(evt: MouseEvent<HTMLButtonElement>) {
evt.preventDefault();
evt.stopPropagation();
setOpen(open => !open);
}
function handleKeyDown(evt: KeyboardEvent<HTMLButtonElement>) {
if (evt.key === "Enter") {
setOpen(true);
}
}
const dropdown = (
<FocusTrap active={open}>
<QueryColumnPicker
query={query}
stageIndex={stageIndex}
columnGroups={columnGroups}
onSelect={onChange}
onClose={handleClose}
checkIsColumnSelected={item => item.column === value}
width="100%"
/>
</FocusTrap>
);
const text = useMemo(() => {
if (!value) {
return t`Select a column...`;
}
const info = Lib.displayInfo(query, stageIndex, value);
return info.longDisplayName;
}, [value, query, stageIndex]);
return (
<Input.Wrapper
label={label}
styles={{
root: { width: "100%" },
label: {
marginBottom: theme.spacing?.xs,
fontSize: theme.fontSizes?.md,
color: color("text-medium"),
},
}}
>
<Popover
opened={open}
onClose={handleClose}
onOpen={handleOpen}
closeOnEscape
closeOnClickOutside
width="target"
returnFocus
>
<Popover.Target>
<Button
ref={button}
onMouseDownCapture={handleButtonClick}
onKeyDown={handleKeyDown}
fullWidth
classNames={{
root: classNames(styles.root, open && styles.open),
inner: styles.button,
}}
rightIcon={<Icon name="chevrondown" style={{ height: 14 }} />}
>
{text}
</Button>
</Popover.Target>
<Popover.Dropdown setupSequencedCloseHandler onBlur={handleBlur}>
{dropdown}
</Popover.Dropdown>
</Popover>
</Input.Wrapper>
);
}
export * from "./ColumnPicker";
import type { FormEventHandler } from "react";
import { useMemo, useState } from "react";
import { t } from "ttag";
import {
Box,
Button,
Card,
Flex,
Icon,
ScrollArea,
Stack,
Title,
} from "metabase/ui";
import * as Lib from "metabase-lib";
import type { ColumnAndSeparator } from "../../types";
import {
formatSeparator,
getDefaultSeparator,
getDrillExpressionClause,
getExample,
getExpressionName,
getNextColumnAndSeparator,
} from "../../utils";
import { ColumnAndSeparatorRow } from "../ColumnAndSeparatorRow";
import { Example } from "../Example";
/**
* Required to not cut off the outline of focused "x" button
*/
const OVERFLOW_SAFETY_MARGIN = 16;
interface Props {
column: Lib.ColumnMetadata;
query: Lib.Query;
stageIndex: number;
onSubmit: (query: Lib.Query) => void;
}
export const CombineColumnsDrill = ({
column,
query: originalQuery,
stageIndex: originalStageIndex,
onSubmit,
}: Props) => {
const columnInfo = Lib.displayInfo(originalQuery, originalStageIndex, column);
const { query, stageIndex } = Lib.asReturned(
originalQuery,
originalStageIndex,
);
const expressionableColumns = Lib.expressionableColumns(query, stageIndex);
const defaultSeparator = getDefaultSeparator(column);
const [isUsingDefaultSeparator, setIsUsingDefaultSeparator] = useState(true);
const [columnsAndSeparators, setColumnsAndSeparators] = useState([
{
column: expressionableColumns[0],
separator: defaultSeparator,
},
]);
const expressionClause = useMemo(
() => getDrillExpressionClause(column, columnsAndSeparators),
[column, columnsAndSeparators],
);
const example = useMemo(() => {
return getExample(column, columnsAndSeparators);
}, [column, columnsAndSeparators]);
const handleChange = (index: number, change: Partial<ColumnAndSeparator>) => {
setColumnsAndSeparators(value => [
...value.slice(0, index),
{ ...value[index], ...change },
...value.slice(index + 1),
]);
};
const handleAdd = () => {
setColumnsAndSeparators(value => [
...value,
getNextColumnAndSeparator(
expressionableColumns,
defaultSeparator,
columnsAndSeparators,
),
]);
};
const handleRemove = (index: number) => {
setColumnsAndSeparators(value => [
...value.slice(0, index),
...value.slice(index + 1),
]);
};
const handleEditSeparators = () => {
setIsUsingDefaultSeparator(false);
};
const handleSubmit: FormEventHandler = event => {
event.preventDefault();
const name = getExpressionName(
query,
stageIndex,
column,
columnsAndSeparators,
);
const newQuery = Lib.expression(query, stageIndex, name, expressionClause);
onSubmit(newQuery);
};
return (
<form onSubmit={handleSubmit}>
<Card maw="100vw" w={474} p="lg">
<Title
mb="lg"
order={4}
>{t`Combine “${columnInfo.displayName}” with other columns`}</Title>
<Stack spacing="lg">
<Stack spacing={12}>
<ScrollArea mx={-OVERFLOW_SAFETY_MARGIN}>
<Box mah="50vh" px={OVERFLOW_SAFETY_MARGIN}>
<Stack spacing="sm">
{columnsAndSeparators.map(({ column, separator }, index) => (
<ColumnAndSeparatorRow
query={query}
stageIndex={stageIndex}
column={column}
columns={expressionableColumns}
index={index}
key={index}
separator={separator}
showLabels={index === 0}
showRemove={columnsAndSeparators.length > 1}
showSeparator={!isUsingDefaultSeparator}
onChange={handleChange}
onRemove={handleRemove}
/>
))}
</Stack>
</Box>
</ScrollArea>
<Flex
align="center"
gap="md"
justify={isUsingDefaultSeparator ? "space-between" : "end"}
>
{isUsingDefaultSeparator && (
<Box>
<Button p={0} variant="subtle" onClick={handleEditSeparators}>
{t`Separated by ${formatSeparator(defaultSeparator)}`}
</Button>
</Box>
)}
<Button
leftIcon={<Icon name="add" />}
p={0}
variant="subtle"
onClick={handleAdd}
>
{t`Add column`}
</Button>
</Flex>
</Stack>
<Example example={example} />
<Flex align="center" gap="md" justify="end">
<Button type="submit" variant="filled">
{t`Done`}
</Button>
</Flex>
</Stack>
</Card>
</form>
);
};
export * from "./CombineColumnsDrill";
.scrollArea {
white-space: pre;
}
import { t } from "ttag";
import { Card, ScrollArea, Stack, Text } from "metabase/ui";
import S from "./Example.module.css";
interface Props {
example: string;
}
export const Example = ({ example }: Props) => {
return (
<Stack spacing="sm">
<Text color="text-medium" lh={1} weight="bold">{t`Example`}</Text>
<Card
bg="bg-light"
className={S.scrollArea}
component={ScrollArea}
p="sm"
radius="xs"
shadow="none"
withBorder
>
<Text
size="sm"
data-testid="combine-column-example"
variant="monospace"
>
{example}
</Text>
</Card>
</Stack>
);
};
export * from "./Example";
export * from "./ColumnAndSeparatorRow";
export * from "./ColumnPicker";
export * from "./CombineColumnsDrill";
export * from "./Example";
export * from "./combine-columns-drill";
import type * as Lib from "metabase-lib";
export type ColumnAndSeparator = {
separator: string;
column: Lib.ColumnMetadata;
};
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment