An AI-assisted agency in thirty days: build a pilot you can evaluate

Article prepared with artificial intelligence assistance. The sources and method are indicated in the text.

Before the first test: choose the right difficulty

The team must name a recurring task and an observable difficulty. Adapting a creation to several formats is not the same project as bringing together a reporting or exploring ideas. The pilot is better off staying limited to make his results interpretable. There is a need for authorized elements, a manager and a comparison method. The thirty-day schedule below is a proposal for organization, not a promise of transformation or productivity gain.

First week: establish the starting point

Describe the current method, the people involved and the usual causes of recovery. Take time on a representative example, with the expected level of quality. Then choose the information needed to test and check their conditions of use. The team should know what data can be shared with the chosen tool and which should remain outside the test. These decisions precede the drafting of the guidelines.

Week 2: Compare multiple responses

Prepare a small set of situations and keep results, including failures. Those who evaluate must share some criteria: accuracy, respect for the brief, possible resumption and final quality. Comparison should not reward only a seductive presentation. Note the necessary corrections and explain the reasons for rejection. A case where the tool recognizes a limit may be more useful than a very assured but incorrect response.

Third week: testing transmission

Ask another person to resume the task with the pilot's documentation. This step reveals the knowledge that remains implicit and the instructions that only work for their author. The protocol must specify the entries, the expected result, the checks and the situations where it is returned to the usual method. The competence is best demonstrated when it can be explained and transmitted.

Fourth week: decide and document

The balance sheet brings together the full time, the usable versions, the errors and the limits. Three possible suites are possible: to continue on the same perimeter, to modify the device or to stop it. Abandoning a solution can be a useful learning if it avoids an investment that is not suitable. The decision must be linked to the initial problem. It should not be changed to match the result with the enthusiasm generated at the launch.

Pilot's decision sheet

The role of management

Management must set aside time for learning and review, not just to ask for more production. People must be able to report a difficulty without the pilot being transformed into a global assessment of their professional value. The skills of judgment, control and transmission deserve to be observed. A team may then decide whether it has the necessary conditions to expand the perimeter.

A management framework to be adapted

NIST offers a risk management profile dedicated to generative artificial intelligence. Our schedule does not reproduce a NIST procedure; it translates the subject into a plan for an experiment for a communication team. Depending on the sensitivity of the project, additional checks and skills will be required. The pilot must remain re-readable and retain a possibility of manual re-run for critical steps.

Sources and method

• NIST, Generative AI Profile AI 600-1

The recommendations and grids of this article are proposed by Le Fils de Pub. The constructed examples are indicated; the advertisements and results attributed to an organization remain those of this source. Article prepared with artificial intelligence assistance.