VAL-150 Conducting an Electronic Record and Signature Assessment

Be sure to differentiate between “signature” and “identification”.

If the intent is to use the applied identification to authenticate the electronic record, the n identification is an electronic signature.

If the intent is to merely identify who did something, then the identification is not an electronic signature.

A question to help determine if it’s a “signature” or “identification” is “IF I sign ‘Jack Handy’, does that mean I have attested that I did or saw something, or that I’m authorizing some action?”

If you have to sign the paper copy, you have to sign the electronic copy.

Whether or not to use electronic signatures is not the system owner’s choice.  The choice is whether to use electronic records or not.  That choice (plus the predicate rules) dictates whether electronic signatures are required or not.

A question to help determine if signatures are required is “If these were printed out, would you need to sign them?”

– Does the predicate rule require signatures on the record?

– Does a company policy require signatures on the record?

– Identify every display screen and report generated by the computerized system where an electronic signature is represented.  Each occurrence should be separately assessed for compliance.

When was it last validated or revalidated?

Does an organization-specific validation standard exist?

Is there documentation to support the validation (list and review at end of the interview if time permits)

Was an established software development life cycle used?

Does a requirements document exist?

Does a design document exist?

Have code reviews been conducted?

If developed in-house, has developer testing been conducted?

If vendor supplied, has an audit of the vendor been conducted?

Has System Testing been conducted?

Has User Acceptance Testing been conducted?

Has Installation Qualification Testing been conducted?

Has Operational Qualification Testing been conducted?

Has Performance Qualification Testing been conducted?

Has formal training been performed?

Is there a support plan?

If this is a legacy system, has a Part 11 assessment been conducted?

Does a Change Control procedure exist? Does evidence exist that it was followed? Does it cover changes to all system components?

Are these documents kept in electronic format? If so, is the document management system Part 11 compliant?

Did validation include testing that the system discerns invalid records (i.e. invalid field entries, fields left blank that should contain data, values outside of limits, ASCII characters in numeric-only fields, etc)?

Can a copy of a single record (in electronic format) be supplied to an inspector? In paper format?

Can a copy of the entire database (in electronic format) be supplied to an inspector?

Are procedures in place to describe HOW to accomplish these inspection tasks?

Are procedures in place to define what format the electronic records will be provided?

Is there test evidence that the process works?

Are records protected on the system to prevent unauthorized modification or deletion?

Are data files written to a protected directory or database table such that only personnel with high-level access privileges can access the data files?

Do system users have access to the data files or database records, such that they could inadvertently or intentionally modify or delete data files?

Has any capacity planning been performed?

—————————————————————————————

Is there a written records retention policy?

Does it include electronic records?

Does it include audit trails?

Does a procedure exist for the backup and restore process?

Does a procedure for data archiving exist? If not, is all data kept on-line?

Does the SOP and actual practice ensure the archived data is controlled and maintained for the required retention period?

Are any backups and/or archives duplicated (e.g., to create off-site backups/archives for disaster recovery)? How is this media protected?

Is the metadata stored with the archived data?

Is virus software loaded and regularly updated to prevent viruses corrupting data?

This requirement refers to both logical and physical access.

Is a username/password (or other logical security) required to access the system?

Is there a security SOP that covers physical and logical security, access authorization, modification, disabling/deleting periodic checking of access, approval by System Owner?

Does an Account Management procedure exist?

Does the system have the ability to generate historical access lists?

Is there test evidence that the access is restricted?

—————————————————————————————

What controls limit physical access to the system?

Is there firewall protection to prevent authorized access from the Internet?

For each type of record in the system, please address the following questions :

Does the system generate automatic, electronic audit trail information (who, what, when)?

Does the audit trail include the reason for change (if required by the predicate rule)?

Is the audit trail function always ON, or is it turned OFF and ON manually?  If manual, who and what triggers audit trail recording?  Does it get turned on early enough in the process?  Is it reliable (i.e., can they forget to turn it on)?

Does the audit trail capture every user action that creates, modified, or deletes records, without exceptions?

When information is changed, does the audit trail record/save the previous value?

Are audit trail entries made at the time the action/operation was conducted electronically?

Is the audit trail ever monitored or reviewed to detect possible misuse or unauthorized activity?

Is it possible to reconstruct events (delete, modify, etc) to any point in time by only using the audit trail information and the original record?

Are electronic audit trails (all or any part) readily available for FDA review and copying? In paper format?

Does the audit trail contain date and time stamps? Can time local to the activity be derived?

Are meaningful units of time chosen in terms of documenting human actions?  (For example, seconds might be used in a data collection system while minutes might be appropriate for a document management system.)

—————————————————————————————

Is the audit trail completely transparent to, and outside the control and access of, the user?

How is audit trail data protected from accidental or intentional modification or deletion?

System Administrators and DBA’s typically make changes. Do those changes have audit trails? If not, do procedural controls exist over use of such administrator tools?

Does the records retention program cover audit trails?

Are electronic audit trails kept for at least as long as their respective electronic records?

What ensures that the system time and date are correct?  How frequently are the time and date synchronized with a reliable source?

Can users readily change the system time/date?

Are time/date stamps applied by the local workstation or by a server (or equivalent)?

Is there test evidence for the audit trail functionality?

Are there sequences of operations, or sequential events, or sequential data entry, that is important to this system?

If so, what are they?

If so, how does the system ensure that steps are followed in the correct sequence?

Does test evidence exist to demonstrate the operational checks?

This requirement refers to functional access once a user logs into the system.

Are there different levels of access based on user responsibilities?  If so, what are they?

Is there an SOP describing how these assigned, documented and controlled?

What process is followed to grant a new user access to the system, or to change privileges for an existing user?  Is it documented?

Are levels of access periodically reviewed?

Are authority checks used to ensure that only authorized individuals can use the system?

Are authority checks used to ensure that only authorized individuals can electronically sign a record?

Are authority checks used to ensure that only authorized individuals can access the operation or computer system input or output devices?

Are authority checks used to ensure that only authorized individuals can alter a record?

Are authority checks used to ensure that only authorized individuals can perform the operation at hand?

Does test evidence exist to demonstrate the use of the authority checks?

Is it necessary to ensure that the data source is identified? If so, what are they? If so, how are they identified?

Example – console commands for a server are limited to the console station

Example – modem access may be verified to ensure the identify of the caller

Are there SOPs, company requirements, job descriptions, etc., that describe minimum education requirements and/or work experience for system developers? Support staff?

For internal persons, is there evidence that they are qualified for their job? (This requirement may be met with CVs, job descriptions, training records, and a training procedure that is followed)

For external persons, is there evidence that they are qualified to perform the work for which they were hired? (This requirement may be met by having their resume on file)

For CROs, is there evidence that they are qualified to perform the work for which they were hired?

Is there an SOP covering user training?

Is there evidence of user training?

—————————————————————————————

Were vendor’s qualifications reviewed during a vendor audit? If so, was the result successful and is it documented?

Is there a list of system documentation related to the development of the system that exists (e.g., requirements, design specifications, training materials, etc.)?

Is the system documentation maintained by Site revision control so changes can be determined and the history of the documents is obvious?

Is access to the Design documentation restricted?

Are these documents kept in electronic format? If so, is the document management system Part 11 compliant?

Is document encryption (or an alternate technology) used to protect the confidentiality of the electronic records on the system?

Are digital signatures (or an alternative technology) use to protect the authenticity and integrity of the electronic records on the system?

Is there a written procedure that describes user responsibilities for the use of computerized systems?

Does it include not sharing passwords, periodically changing passwords, not using easy to guess passwords?

Does it include not installing unapproved software and running virus protection software?

Does user acceptance/approval in writing that acknowledges their understanding that their electronic signature is the legally binding equivalent of the handwritten signature exist?

Is the full name (first and last) is displayed? The printed name cannot be the User ID.

Is the meaning of the signature included?

Is the Date and Time included? Precision of time is based on risk. For example, the time might be reported in seconds for a data collection system. The time might be reported in minutes for a document management system.

Is the manifestation information under the same controls as for electronic records?

Can the local time be derived if the system runs across time zones?

Where is the time taken from? Is it protected from change by the user?

Are the 4 components in the screen displays and reports?

Does test evidence exist?

Is the electronic signature linked to the electronic record?

Is the transfer of the signature to another record prevented?

Is the record protected to prevent changes after signing or to force re-signing?

Are signature changes recorded in the audit trail?

Does test evidence exist?

Is there a policy or procedure explicitly stating that each assigned electronic signature is unique to one person?

Does the system enforce unique username/id?

Is there a policy or procedure that explicitly states that electronic signatures shall not be reused by or reassigned to anyone else?

Does test evidence exist?

Has the contractor or temporary employee been cleared by Security or Human Resources to enter the workplace?

Are controls in place to ensure that fake identities can be discerned with high reliability?

Are controls in place to verify that requestors are authorized to make requests for an e-signature (i.e., on behalf of themselves or another user)?

Are individuals required to show ID when they are given their electronic signatures/

Are individuals requested to verify their identity if they forget their password?

The certification letter has been sent.

Is there documentation to support that individuals understand that electronic signatures are legally binding?

What format is the additional testimony (training, signing of “evidence of understanding”?

Is there documented training for persons using electronic signatures?

Is there a procedure that states that the electronic signature is the legal binding equivalent of the handwritten signature?

Initial log on to the system requires the execution of the identification code and password. This is not a signature.

The combination of these two components must be unique.

The first signing requires both components.

Is there a definition for a continuous session?

Subsequent signings in the same session, requires only the password (which is

When executing more than one signing not performed during a single continuous period, both the user id and password are required?

If, when resetting the account on some systems, a “default” password is assigned, is the user forced to change the password immediately upon log on?

When an identification code and password are used as the electronic signature, is the password unknown to everyone, including the System Administrator?

Are system tools used that might allow a System Administrator to falsify electronic records and/or electronic signatures? If so, are there procedures in place to ensure adequate controls over these activities?

Does the system/workstation log-out after a period of inactivity?

Do procedures and training reinforce that non-biometric electronic signatures must not be shared or loaned?

Are safeguards in place that prevent one person from forging another person’s electronic signature?

Is the non-biometric electronic signature only used by the genuine owner?

Are the controls in place to require the collaboration of two or more individuals when some one other than the genuine owner attempts to use it?

ID Code and Password Only Questions

Does a corporate policy exist?

Is uniqueness maintained historically?

Does the system check for duplicate Ids?

Does a written procedure exist?

Does the computerized system include functionality that requires users to periodically change their passwords (password aging)?

Is there a manual procedure that requires users to periodically change their passwords?

Are the controls built into the system?

Is there a procedure or system function that revokes sign-on privileges when an incorrect combination of identification and password are repeatedly entered?

Has testing been conducted to ensure that “inactive” user accounts cannot be activated by unauthorized persons?

Are there procedures and appropriate training to assure that users understand that passwords are not to be shared?

Does the system create alert messages for unauthorized access attempts (e.g., access violations)?

Is access frozen after a number of unsuccessful attempts to log in?

Are “attempts at unauthorized use” defined?

Are potential break-in attempts monitored in real-time?

Is access violation reporting monitoring and escalation addressed in a SOP?

Is “immediate and urgent” defined?  Is the procedure and timing for notifying management defined?

Does the procedure describe the security group’s responsibility and required activities when notified of possible security breaches?

Does test evidence exist?

Does a written procedure for loss management exist

Is there a procedure to describe how temporary replacements are handled?

Is there a procedure that requires both initial and periodic testing of these devices?

Is initial testing of the devices conducted to ensure that they are tamper-proof and reliable?

Is periodic re-testing of devices conducted prior to putting new stock and/or models into service?

Are there testing steps to ensure that devices operate within the manufacturer’s operating parameters and functional tolerances?

Does test evidence exist?

A properly designed and implemented biometric-based electronic signature system makes it unlikely that any electronic signature could be falsified.

Does testing evidence exist?

If image files are used, are they stored in a manner in which they could not be copied and applied elsewhere?