PropScreen Overview
Short for time? Here is an overview of what Propscreen is!
PropScreen is a contraction of the words "proprietary" and "screen" and is named such because the primary function of PropScreen is to screen outputs from LLMs that have been trained on proprietary organizational data or other data that an organization has deemed to be sensitive. These outputs are screened in a manner such that if a user is going to receive a response that discloses data that is in violation of organizational policy, the response is interdicted and the user receives an error message instead. In short, Propscreen can detect when sensitive information is present in a model's response that is going to be sent to the user and block that response from being sent. After each response is scanned, a log entry of the interdiction event is created as stored for that organization's security team to audit.
In its current iteration, Propscreen has two primary functionalities: the interdiction of sensitive information and the generation of logs based on those interdictions. Sensitive information interdiction occurs by passing the LLM's responses through a series of three possible checks: a check against general schemas of sensitive information by means of a Named Entry Recognition model provided by LLM Guard, a check of the model response against a database of words set by the organization trigger further inquiry, and finally if either one of the two previous checks yields a match, a final check of the hashes of the tokens in the response against a database of the hashed sensitive information.
If a match is detected between at least one of the token hashes in the response and a hash in the database, then PropScreen's ability to interdict the response is used and the user receives an error message instead of the response.
The contextual database and hashes database empower an organization to deterministically prevent sensitive information specific to the company form being disclosed to unauthorized parties. This information does not need to follow a specific pattern or fit into a specific schema. As long as the data is present in the response of the model, it can be identified and interdicted.
The utilization of a hashed database may seem like a myopic approach to solve the problem, however tweaking an existing model to identify sensitive information is currently a costly project, and the model would have to be retrained in order to effectively recognize new sets of classified data. PropScreen offers a more dynamic and efficient solution to this approach. The utilization of a sensitive information context scanner and a database of context words reduces the amount of responses that are sent through the hashed database checks, reducing compuational load.
Last updated